cpsc 340 tutorial assignment 5 part 2schmidtm/courses/340-f15/t5b.pdf · assignment 5 part 2...
TRANSCRIPT
![Page 1: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/1.jpg)
CPSC 340 TutorialAssignment 5 part 2
Questions 3 to 5
![Page 2: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/2.jpg)
Multi-Dimensional Scaling (MDS)
![Page 3: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/3.jpg)
Multi-Dimensional Scaling (MDS)
![Page 4: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/4.jpg)
Multi-Dimensional Scaling (MDS)
● Recall: principal component analysis (PCA) projects d-dimensional data points to a hyperplane orthogonal to the directions of maximal variance.
Eigenvectors of (directional vectors)
Covariance matrix
![Page 5: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/5.jpg)
Multi-Dimensional Scaling (MDS)
● Recall: principal component analysis (PCA) projects d-dimensional data points to a hyperplane orthogonal to the directions of maximal variance
● PCA preserves covariance between the data points
![Page 6: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/6.jpg)
Multi-Dimensional Scaling (MDS)
● MDS projects data points to a space where similar data points are clustered together
● MDS preserves distances between points
![Page 7: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/7.jpg)
Multi-Dimensional Scaling (MDS)
● MDS preserves distances between points
Original distances but in d-dimensional space Derives k-dimensional data points such that the original distances are preserved
Optimizing for Z
![Page 8: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/8.jpg)
Multi-Dimensional Scaling (MDS)
● We want● where, for example,
Original distances but in d-dimensional space Derives k-dimensional data points such that the original distances are preserved
![Page 9: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/9.jpg)
Question 3: exampleMDS exampleMDS.m file
![Page 10: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/10.jpg)
Question 3: exampleMDS Figure 1 - Displays matrix X
X consists of values 0 and 1
blue is 0; red is 1
n samples
d features
![Page 11: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/11.jpg)
Question 3: exampleMDSFigure 2 - Displays the projection of X ontothe first two Principal components
Z1
Z2
![Page 12: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/12.jpg)
Question 3: exampleMDSFigure 3 - Displays the datasets in termsof the two latent features obtainedfrom MDS
Z1
Z2
![Page 13: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/13.jpg)
Question 3: exampleMDSFigure 3 - Displays the datasets in termsof the two latent features obtainedfrom MDS
Z1
Z2
![Page 14: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/14.jpg)
Initializes Z using PCA
Computes the distances between each pair of samples
MDS
![Page 15: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/15.jpg)
MDS function value
MDS
![Page 16: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/16.jpg)
MDS function value
MDS
![Page 17: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/17.jpg)
MDS function value
MDS
![Page 18: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/18.jpg)
MDS
MDS function value
![Page 19: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/19.jpg)
MDS function value
MDS gradient w.r.t zi and zj
![Page 20: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/20.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
![Page 21: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/21.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
?
![Page 22: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/22.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
![Page 23: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/23.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
?
![Page 24: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/24.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
![Page 25: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/25.jpg)
MDS function value
MDS gradient w.r.t zi and zj
Derivative
![Page 26: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/26.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping objective function
MDS objective function
![Page 27: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/27.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping objective function
MDS objective function
![Page 28: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/28.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping objective function
MDS objective function
![Page 29: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/29.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping objective function
MDS objective function
Make changes here
Recall:
![Page 30: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/30.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping objective function
MDS objective function
Make changes here
Recall:
?
![Page 31: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/31.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping gradient function
MDS gradient function
?
![Page 32: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/32.jpg)
Question 3.1: visualizeSammon
Sammon’s mapping gradient function
MDS gradient function
?
?Make changes here
![Page 33: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/33.jpg)
Question 3.2: ISOMAP
![Page 34: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/34.jpg)
Question 3.2: ISOMAP
x1
x2
x3
x2
x1
x3
ISOMAP using 1-nearest neighborMulti-dimensional scaling
![Page 35: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/35.jpg)
Question 3.2: ISOMAP
x1
x2x2
x1
x3
ISOMAP using 1-nearest neighborMulti-dimensional scaling
D(x1, x3)
x3
D(x1, x3)
![Page 36: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/36.jpg)
Question 3.2: ISOMAP
x1
x2x2
x1
x3
ISOMAP using 1-nearest neighborMulti-dimensional scaling
D(x1, x3)
x3
D(x1, x3)
![Page 37: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/37.jpg)
Question 3.2: ISOMAP● If x is d-dimensional,
○ then ISOMAP with n-nearest neighbor is MDS○ Otherwise the only difference is in the distance function
![Page 38: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/38.jpg)
● If x is d-dimensional,○ then ISOMAP with d-nearest neighbor is MDS○ Otherwise the only difference is in the distance function
ISOMAP: Make changes here
![Page 39: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/39.jpg)
● If x is d-dimensional,○ then ISOMAP with d-nearest neighbor is MDS○ Otherwise the only difference is in the distance function
x2
x1
x3
D(x1, x3)
ISOMAP: 1) Find k-nearest neighbors
The ‘k’ points that are closest to each data point (see KNN)
![Page 40: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/40.jpg)
ISOMAP: 2) Create n x n zero matrix G (the adjacency graph)
s.t.
x2
x1
x3
D(x1, x3)
![Page 41: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/41.jpg)
ISOMAP: 2) Create n x n zero matrix G (the adjacency graph)
s.t.
x2
x1
x3
D(x1, x3)
3) Use the Djikstra function to get the shortest distance between each point i and j
![Page 42: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/42.jpg)
ISOMAP: 2) Create n x n zero matrix G (the adjacency graph)
s.t.
x2
x1
x3
D(x1, x3)
3) Use the Djikstra function to get the shortest distance between each point i and j
The adjacency matrixUpdate the distance matrix
![Page 43: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/43.jpg)
Q4: Visualizing a neural net for 1D regression
![Page 44: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/44.jpg)
Original performance
![Page 45: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/45.jpg)
Original performance
Target performance
![Page 46: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/46.jpg)
Number of hidden neurons
Improve the performance of your neural network
The size of the step to take when updating ‘w’
nParams: the total number of variables
i: the index of the next sample ‘x’ to be choseng: the gradient of that sample with respect to ‘w’
t: the epoch number
![Page 47: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/47.jpg)
Adjust the number of hidden neurons(the more hidden neurons, the more powerful your model will be, but couldcause overfitting)
Improve the performance of your neural network
Adjust the step size● Large step size can cause oscillations in your function value● Small step size can cause slow training
![Page 48: CPSC 340 Tutorial Assignment 5 part 2schmidtm/Courses/340-F15/t5b.pdf · Assignment 5 part 2 Questions 3 to 5. Multi-Dimensional Scaling (MDS) Multi-Dimensional Scaling (MDS) Multi-Dimensional](https://reader034.vdocuments.us/reader034/viewer/2022042712/5f9e0004fa2e6979a832ea1e/html5/thumbnails/48.jpg)
Original performance
Target performance