1
UNC Stat amp OR
DWD in Face Recognition (cont)
Interesting
summary
Jump between
means
(in DWD direction)
Clear separation of
Maleness vs
Femaleness
2
UNC Stat amp OR
DWD in Face Recognition (cont)
Fun Comparison
Jump between
means
(in SVM direction)
Also distinguishes
Maleness vs
Femaleness
But not as well as
DWD
3
UNC Stat amp OR
DWD in Face Recognition (cont)
Analysis of difference Project onto normals SVM has ldquosmall gaprdquo (feels noise
artifacts) DWD ldquomore informativerdquo (feels real
structure)
HDLSS Discrimrsquon Simulations
Main idea
Comparison of
bull SVM (Support Vector Machine)
bull DWD (Distance Weighted Discrimination)
bull MD (Mean Difference aka Centroid)
Linear versions across dimensions
HDLSS Discrimrsquon Simulations
Overall Approachbull Study different known phenomena
ndash Spherical Gaussiansndash Outliersndash Polynomial Embedding
bull Common Sample Sizes
bull But wide range of dimensions25 nn
16004001004010d
HDLSS Discrimrsquon Simulations
Spherical Gaussians
HDLSS Discrimrsquon Simulations
Spherical Gaussiansbull Same setup as beforebull Means shifted in dim 1 onlybull All methods pretty goodbull Harder problem for higher dimensionbull SVM noticeably worsebull MD best (Likelihood method)bull DWD very close to MDbull Methods converge for higher
dimension
221
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Main idea
Comparison of
bull SVM (Support Vector Machine)
bull DWD (Distance Weighted Discrimination)
bull MD (Mean Difference aka Centroid)
Linear versions across dimensions
HDLSS Discrimrsquon Simulations
Overall Approachbull Study different known phenomena
ndash Spherical Gaussiansndash Outliersndash Polynomial Embedding
bull Common Sample Sizes
bull But wide range of dimensions25 nn
16004001004010d
HDLSS Discrimrsquon Simulations
Spherical Gaussians
HDLSS Discrimrsquon Simulations
Spherical Gaussiansbull Same setup as beforebull Means shifted in dim 1 onlybull All methods pretty goodbull Harder problem for higher dimensionbull SVM noticeably worsebull MD best (Likelihood method)bull DWD very close to MDbull Methods converge for higher
dimension
221
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Overall Approachbull Study different known phenomena
ndash Spherical Gaussiansndash Outliersndash Polynomial Embedding
bull Common Sample Sizes
bull But wide range of dimensions25 nn
16004001004010d
HDLSS Discrimrsquon Simulations
Spherical Gaussians
HDLSS Discrimrsquon Simulations
Spherical Gaussiansbull Same setup as beforebull Means shifted in dim 1 onlybull All methods pretty goodbull Harder problem for higher dimensionbull SVM noticeably worsebull MD best (Likelihood method)bull DWD very close to MDbull Methods converge for higher
dimension
221
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Spherical Gaussians
HDLSS Discrimrsquon Simulations
Spherical Gaussiansbull Same setup as beforebull Means shifted in dim 1 onlybull All methods pretty goodbull Harder problem for higher dimensionbull SVM noticeably worsebull MD best (Likelihood method)bull DWD very close to MDbull Methods converge for higher
dimension
221
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Spherical Gaussiansbull Same setup as beforebull Means shifted in dim 1 onlybull All methods pretty goodbull Harder problem for higher dimensionbull SVM noticeably worsebull MD best (Likelihood method)bull DWD very close to MDbull Methods converge for higher
dimension
221
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Outlier Mixture
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Outlier Mixture80 dim 1 other dims 020 dim 1 plusmn100 dim 2 plusmn500
others 0bull MD is a disaster driven by outliersbull SVM amp DWD are both very robustbull SVM is bestbull DWD very close to SVM (insigrsquot
difference)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Wobble Mixture
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Wobble Mixture80 dim 1 other dims 020 dim 1 plusmn01 rand dim plusmn100
others 0bull MD still very bad driven by outliersbull SVM amp DWD are both very robustbull SVM loses (affected by margin push)bull DWD slightly better (by wrsquoted
influence)bull Methods converge for higher
dimensionIgnore RLR (a mistake)
221
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Nested Spheres
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon SimulationsNested Spheres
1st d2 dimrsquos Gaussian with var 1 or C2nd d2 dimrsquos the squares of the 1st
dimrsquos(as for 2nd degree polynomial
embedding) bull Each method best somewherebull MD best in highest d (data non-
Gaussian)bull Methods not comparable (realistic)bull Methods converge for higher
dimensionbull HDLSS space is a strange place
Ignore RLR (a mistake)
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon SimulationsConclusions
bull Everything (sensible) is best sometimes
bull DWD often very near bestbull MD weak beyond Gaussian
Caution about simulations (and examples)
bull Very easy to cherry pick best onesbull Good practice in Machine Learning
ndash ldquoIgnore method proposed but read paper for useful comparison of
othersrdquo
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Caution There are additional players
Eg Regularized Logistic Regression
looks also very competitive
Interesting Phenomenon
All methods come together
in very high dimensions
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
HDLSS Discrimrsquon Simulations
Can we say more about
All methods come together
in very high dimensions
Mathematical Statistical Question
Mathematics behind this
(will answer later)
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
SVM amp DWD Tuning Parameter
Main Idea
Handling of Violators (ldquoSlack Variablesrdquo)
Controlled by Tuning Parameter C
Larger C Try Harder to Avoid Violation
SVM Tuning ParameterRecall Movie for SVM
SVM Tuning ParameterRecall Movie for SVM
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
(Can be Effective
But Takes Time
Requires Expertise)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
DWD 100 median pairwise
distance
(Surprisingly Useful Simple Answer)
SVM 1000
(Works Well Sometimes Not Others)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
(Works Well for DWD
Less Effective for SVM)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
Measure Classification Error Rate
Leaving Some Out (to Avoid Overfitting)
Choose C to Minimize Error Rate
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
(Very Popular ndash Useful for SVD
But Comes at Computational Cost)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
DWD 100 median pairwise
distance
(Surprisingly Useful Simple Answer)
SVM 1000
(Works Well Sometimes Not Others)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
(Works Well for DWD
Less Effective for SVM)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
Measure Classification Error Rate
Leaving Some Out (to Avoid Overfitting)
Choose C to Minimize Error Rate
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
(Very Popular ndash Useful for SVD
But Comes at Computational Cost)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
(Works Well for DWD
Less Effective for SVM)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
Measure Classification Error Rate
Leaving Some Out (to Avoid Overfitting)
Choose C to Minimize Error Rate
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
(Very Popular ndash Useful for SVD
But Comes at Computational Cost)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
Measure Classification Error Rate
Leaving Some Out (to Avoid Overfitting)
Choose C to Minimize Error Rate
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
(Very Popular ndash Useful for SVD
But Comes at Computational Cost)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
(Very Popular ndash Useful for SVD
But Comes at Computational Cost)
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
SVM amp DWD Tuning Parameter
Possible Approaches
bull Visually Tuned
bull Simple Defaults
bull Cross Validation
bull Scale Space
(Work with Full Range of Choices)
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Use Image Features as Before
(Recall from Transformation Discussion)
Paper Miedema et al (2012)
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 26
Clinical diagnosis
BackgroundIntroduction
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 27
Image Analysis of Histology Slides
GoalBackground
Melanoma
Image wwwmelanomaca
Benign1 in 75 North Americans will develop a malignant melanoma in their lifetime
Initial goal Automatically segment nucleiChallenge Dense packing of nucleiUltimately Cancer grading and patient survival
Image melanomablogsomecom
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 28
Feature Extraction
Features from Cell NucleiFeature Extraction
Extract various features based on color and morphology
Example ldquohigh-levelrdquo concepts
bull Stain intensity
bull Nuclear area
bull Density of nuclei
bull Regularity of nuclear shape
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 29
Labeled Nuclei
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 30
Nuclear Regions
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Generated by growing nuclei out from boundary
Used for various color and density features Region Stain 2 Region Area Ratio etc
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
March 17 2010 31
Delaunay Triangulation
Features from Cell NucleiFeature Extraction
Conventional Nevus Superficial Spreading Melanoma
Triangulation of nuclear centers
Used for various density features Mean Delaunay Max Delaunay etc
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Databull Study Differences Between
(Malignant) Melanoma amp (Benign) Nevi
Explore with PCA View
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
PCA
View
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To DWD
Direction
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To DWD
Direction
Orthogonal
PCs
Avoid
Strange
Projections
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Return
To
PCA
View
And
Focus
On
Subtypes
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Focus
On
Subtypes
Melanoma 1
Sev Dys
Nevi
Gray Out
Others
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To
Pairwise
Only
PCA
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate
To DWD
amp Ortho PCs
Better
Separation
Than Full
Data
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Full Data
DWD
Direction
ldquoGoodrdquo
Separation
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Developed in WWII History in
Green and Swets (1966)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Approach from Signal Detection
Receiver Operator Characteristic
(ROC) Curve
Good Modern Treatment
DeLong DeLong amp Clarke-Pearson (1988)
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Challenge
Measure ldquoGoodness of Separationrdquo
Idea For Range of Cutoffs
Plot
Proprsquon +1rsquos Smaller than Cutoff
Vs
Proprsquon -1s Smaller than Cutoff
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Aim
Quantify
ldquoOverlaprdquo
Approach
Consider
Series
Of Cutoffs
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Approach
Consider
Series
Of Cutoffs
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
X-coord
Is Proprsquon
Of Reds
Smaller
Y-coord
Is Proprsquon
Of Blues
Smaller
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Slide
Cutoff
To
Trace
Out
Curve
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Better
Separation
Is ldquoMore
To Upper
Leftrdquo
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Summarize
amp Compare
Using
Area
Under
Curve
(AUC)
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Perfect
Separation
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Very
Slight
Overlap
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Little
More
Overlap
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
More
Overlap
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Much
More
Overlap
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Complete
Overlap
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Complete
Overlap
AUC asymp 05 Reflects ldquoCoin Tossingrdquo
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Toy
Example
Can
Reflect
ldquoWorse
Than Coin
Tossingrdquo
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ROC Curve
Interpretation of AUC
Very Context Dependent
Radiology
ldquogt 70 has Predictive Usefulnessrdquo
Bigger is Better
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Subclass
DWD
Direction
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Full Data
DWD
Direction
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Recall Question
Which Gives Better Separation of
Melanoma vs Nevi
DWD on All Melanoma vs All Nevi DWD on Melanoma 1 vs Sev Dys Nevi
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Full Data
ROC
Analysis
AUC = 093
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
SubClass
ROC
Analysis
AUC = 095
Better
Makes
Intuitive
Sense
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
What About Other Subclasses
Looked at Several
Best Separation Was
Melanoma 2 vs Conventional Nevi
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Full
Data
PCA
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Full
Data
PCA
Gray
Out
All
But
Subclasses
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate to
SubClass
PCA
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
Rotate to
SubClass
DWD
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
Melanoma Data
ROC
Analysis
AUC = 099
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ClusteringIdea Given data bull Assign each object to a class
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data driven
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
nXX
1
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ClusteringIdea Given data bull Assign each object to a classbull Of similar objectsbull Completely data drivenbull Ie assign labels to databull ldquoUnsupervised Learningrdquo
Contrast to Classification (Discrimination)bull With predetermined classesbull ldquoSupervised Learningrdquo
nXX
1
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
ClusteringImportant Referencesbull MacQueen (1967)bull Hartigan (1975)bull Gersho and Gray (1992)bull Kaufman and Rousseeuw (2005)
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringMain Idea for data
Partition indices
among classes
nXX
1
ni 1
KCC 1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition
nXX
1
ni 1
KCC 1
KCC 1
n1
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringMain Idea for data
Partition indices
among classes
Given index sets bull that partition bull represent clusters by ldquoclass meansrdquo
ie (within class means)
nXX
1
ni 1
KCC 1
KCC 1
jCi
ij
j XC
X
1
n1
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringGiven index sets
Measure how well clustered using
Within Class Sum of Squares
KCC 1
2
1
jCi
ji
K
j
XX
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringCommon Variation
Put on scale of proportions (ie in [01])
By dividing ldquowithin class SSrdquo
by ldquooverall SSrdquo
Gives Cluster Index
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster means
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means ClusteringNotes on Cluster Index
bull CI = 0 when all data at cluster meansbull CI small when gives tight clustering
(within SS contains little variation)bull CI big when gives poor clustering
(within SS contains most of variation)bull CI = 1 when all cluster means are same
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
KCC 1
KCC 1
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
K-means Clustering
Clustering Goal
bull Given data
bull Choose classes
bull To miminize
KCC 1
nXX
1
n
ii
Ciji
K
j
K
XX
XX
CCCI j
1
2
2
1
1
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Varying Standard Deviation
bull Varying Mean
bull Varying Proportion
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Multiple local minima (large number)
ndash Maybe disconnected
ndash Optimization (over ) can be trickyhellip
(even in 1 dimension with K = 2)
KCC 1
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Can have 4 (or more) local mins
(even in 1 dimension with K = 2)
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Over changing Classes (moving brsquodry)
bull Multi-modal data interesting effects
ndash Local mins can be hard to find
ndash ie iterative procedures can ldquoget stuckrdquo
(even in 1 dimension with K = 2)
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-
2-means Clustering
Study CI using simple 1-d examples
bull Effect of a single outlier
ndash Can create local minimum
ndash Can also yield a global minimum
ndash This gives a one point class
ndash Can make CI arbitrarily small
(really a ldquogood clusteringrdquo)
- DWD in Face Recognition (cont)
- DWD in Face Recognition (cont) (2)
- DWD in Face Recognition (cont) (3)
- HDLSS Discrimrsquon Simulations
- HDLSS Discrimrsquon Simulations (2)
- HDLSS Discrimrsquon Simulations (3)
- HDLSS Discrimrsquon Simulations (4)
- HDLSS Discrimrsquon Simulations (5)
- HDLSS Discrimrsquon Simulations (6)
- HDLSS Discrimrsquon Simulations (7)
- HDLSS Discrimrsquon Simulations (8)
- HDLSS Discrimrsquon Simulations (9)
- HDLSS Discrimrsquon Simulations (10)
- HDLSS Discrimrsquon Simulations (11)
- HDLSS Discrimrsquon Simulations (12)
- HDLSS Discrimrsquon Simulations (13)
- SVM amp DWD Tuning Parameter
- SVM Tuning Parameter
- SVM amp DWD Tuning Parameter (2)
- SVM amp DWD Tuning Parameter (3)
- SVM amp DWD Tuning Parameter (4)
- SVM amp DWD Tuning Parameter (5)
- SVM amp DWD Tuning Parameter (6)
- SVM amp DWD Tuning Parameter (7)
- Melanoma Data
- Slide 26
- Slide 27
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Melanoma Data (2)
- Melanoma Data (3)
- Melanoma Data (4)
- Melanoma Data (5)
- Melanoma Data (6)
- Melanoma Data (7)
- Melanoma Data (8)
- Melanoma Data (9)
- Melanoma Data (10)
- Melanoma Data (11)
- Melanoma Data (12)
- Melanoma Data (13)
- ROC Curve
- ROC Curve (2)
- ROC Curve (3)
- ROC Curve (4)
- ROC Curve (5)
- ROC Curve (6)
- ROC Curve (7)
- ROC Curve (8)
- ROC Curve (9)
- ROC Curve (10)
- ROC Curve (11)
- ROC Curve (12)
- ROC Curve (13)
- ROC Curve (14)
- ROC Curve (15)
- ROC Curve (16)
- ROC Curve (17)
- ROC Curve (18)
- ROC Curve (19)
- ROC Curve (20)
- ROC Curve (21)
- ROC Curve (22)
- ROC Curve (23)
- Melanoma Data (14)
- Melanoma Data (15)
- Melanoma Data (16)
- Melanoma Data (17)
- Melanoma Data (18)
- Melanoma Data (19)
- Melanoma Data (20)
- Melanoma Data (21)
- Melanoma Data (22)
- Melanoma Data (23)
- Melanoma Data (24)
- Melanoma Data (25)
- Clustering
- Clustering (2)
- Clustering (3)
- Clustering (4)
- Clustering (5)
- K-means Clustering
- K-means Clustering (2)
- K-means Clustering (3)
- K-means Clustering (4)
- K-means Clustering (5)
- K-means Clustering (6)
- K-means Clustering (7)
- K-means Clustering (8)
- K-means Clustering (9)
- K-means Clustering (10)
- K-means Clustering (11)
- K-means Clustering (12)
- 2-means Clustering
- 2-means Clustering (2)
- 2-means Clustering (3)
- 2-means Clustering (4)
- 2-means Clustering (5)
- 2-means Clustering (6)
- 2-means Clustering (7)
- 2-means Clustering (8)
- 2-means Clustering (9)
- 2-means Clustering (10)
- 2-means Clustering (11)
- 2-means Clustering (12)
- 2-means Clustering (13)
- 2-means Clustering (14)
- 2-means Clustering (15)
- 2-means Clustering (16)
- 2-means Clustering (17)
- 2-means Clustering (18)
- 2-means Clustering (19)
- 2-means Clustering (20)
- 2-means Clustering (21)
- 2-means Clustering (22)
- 2-means Clustering (23)
- 2-means Clustering (24)
- 2-means Clustering (25)
- 2-means Clustering (26)
- 2-means Clustering (27)
- 2-means Clustering (28)
- 2-means Clustering (29)
- 2-means Clustering (30)
- 2-means Clustering (31)
- 2-means Clustering (32)
- 2-means Clustering (33)
- 2-means Clustering (34)
- 2-means Clustering (35)
- 2-means Clustering (36)
- 2-means Clustering (37)
- 2-means Clustering (38)
- 2-means Clustering (39)
- 2-means Clustering (40)
- 2-means Clustering (41)
- 2-means Clustering (42)
- 2-means Clustering (43)
- 2-means Clustering (44)
- 2-means Clustering (45)
- 2-means Clustering (46)
- 2-means Clustering (47)
- 2-means Clustering (48)
- 2-means Clustering (49)
- 2-means Clustering (50)
- 2-means Clustering (51)
- 2-means Clustering (52)
- 2-means Clustering (53)
- 2-means Clustering (54)
- 2-means Clustering (55)
- 2-means Clustering (56)
- 2-means Clustering (57)
- 2-means Clustering (58)
- 2-means Clustering (59)
- 2-means Clustering (60)
- 2-means Clustering (61)
- 2-means Clustering (62)
- 2-means Clustering (63)
- 2-means Clustering (64)
- 2-means Clustering (65)
- 2-means Clustering (66)
- 2-means Clustering (67)
- 2-means Clustering (68)
-