Artificial Neural Networks

Artificial Neural Networks

Inspired by biology

A variety of different analytics tasks

Regression, classification, clustering, feature extraction, etc.

Network of simple computing entities

Resurgence in interest recently

Artificial Neural Networks

Inspired by biology

A variety of different analytics tasks

Regression, classification, clustering, feature extraction, etc.

Network of simple computing entities

Resurgence in interest recently

Biological Neuron

Biological Neuron

Biological Neural Network

A simple computing unit

McCulloch-Pitts Unit (1943)

Uses only binary signals : 0,1

Nodes produce only binary results.

Edges Directed, unweighted Excitatory or inhibitory

type transmit exclusively

binary signals

Rule for evaluating the input

Assume that a M-P unit gets inputs x1, x2, , xn through n excitatory edges and inputs y1, y2, , ym through m inhibitory edges.

If m >= 1 and at least one of the yi is 1, the unit is inhibited and the result of the computation is 0.

Otherwise, compute x = x1 + x2 + + xn . If x >= , the result is 1, else the result is 0.

The step function with threshold

Some Boolean functions

A Simple Perceptron (Rosenblatt, 1957)

A simple perceptron is a computing unit with a threshold of such that

q

o x1,, xn( ) =1 if wixi qi=1

n

= -1 otherwise

o x1,, xn( )

Perceptron Training Algorithm

What can a perceptron learn?

Two example problems

XOR Problem (Minsky and Papert, 1969)

What can a perceptron learn?

Perceptrons can learn only linear decision boundaries!

Significance of Representations

Multilayer Neural Network

Standard 3 layer network

Activation functions

Activation functions