neural networks: prediction (i.e. the forward pass) · based on slides and material from geoffrey...

MachineLearning

NeuralNetworks:Prediction

(i.e.theforwardpass)

1BasedonslidesandmaterialfromGeoffreyHinton,RichardSocher,DanRoth,

Yoav Goldberg,ShaiShalev-Shwartz andShaiBen-David,andothers

NeuralNetworks

• Whatisaneuralnetwork?

• Predictingwithaneuralnetwork

• Trainingneuralnetworks

• Practicalconcerns

2

Thislecture

• Whatisaneuralnetwork?

• Predictingwithaneuralnetwork

• Trainingneuralnetworks

• Practicalconcerns

3

Letusconsideranexamplenetwork

4

Wewillusethisexamplenetwork

astointroducethegeneral

principleofhowtomake

predictionswithaneuralnetwork.

Letusconsideranexamplenetwork

5

Namingconventionsforthis

example

• Inputs:x

• Hidden:z

• Output:y

Letusconsideranexamplenetwork

6

Biasfeature,

always1

Namingconventionsforthis

example

• Inputs:x

• Hidden:z

• Output:y

Letusconsideranexamplenetwork

8

Biasfeature,

always1

Sigmoidactivations

Linearactivation

Namingconventionsforthis

example

• Inputs:x

• Hidden:z

• Output:y

Letusconsideranexamplenetwork

9

NamingConventionforWeights

!"#$%,'$'(#)*'_,(-*#

Letusconsideranexamplenetwork

10

NamingConventionforWeights

!"#$%,'$'(#)*'_,(-*#

!./$Fromneuron#0

toneuron#1in

outputlayer

Howtopredictwithaneuralnetwork:The

forwardpass

11

Givenaninputx,howistheoutputpredicted

Theforwardpass

12

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Givenaninputx,howistheoutputpredicted

Theforwardpass

13

97 = 2(!.74 + !/74 6/ + !774 67)

Givenaninputx,howistheoutputpredicted

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Theforwardpass

14

y = !./$ + !//$ 9/ + !7/$ 97output

Givenaninputx,howistheoutputpredicted

97 = 2(!.74 + !/74 6/ + !774 67)

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Theforwardpass

15

y = !./$ + !//$ 9/ + !7/$ 97output

Givenaninputx,howistheoutputpredicted

97 = 2(!.74 + !/74 6/ + !774 67)

z/ = 2(!./4 + !//4 6/ + !7/4 67)

Questions?

Ingeneral,beforevisiting(i.e.computing)thevalueofanode,visitallnodesthatserveasinputstoit.