perceptron networks and vector notation
DESCRIPTION
Perceptron Networks and Vector Notation. CS/PY 231 Lab Presentation # 3 January 31, 2005 Mount Union College. A Multiple Perceptron Network for computing the XOR function. We found that a single perceptron could not compute the XOR function - PowerPoint PPT PresentationTRANSCRIPT
Perceptron Networks and Vector Notation
CS/PY 231 Lab Presentation # 3 January 31, 2005 Mount Union College
A Multiple Perceptron Network for computing the XOR function We found that a single perceptron could not
compute the XOR function Solution: set up one perceptron to detect if x1
= 1 and x2 = 0 set up another perceptron for x1 = 0 and x2 =
1 feed the outputs of these two perceptrons into
a third one that produces an output of 1 if either input is a 1
A Nightmare!
Even for this simple example, choosing the weights that cause a network to compute the desired output takes skill and lots of patience
Much more difficult than programming a conventional computer:
• OR function: if x1 + x2 > 1, output 1; otherwise output 0
• XOR function: if x1 + x2 = 1, output 1; otherwise output 0
There must be a better way….
These labs and demos were designed to show that manually adjusting weights is tedious and difficult
This is not what happens in nature– No creature says, “Hmmm, what weight should I
choose for this neural connection?” Formal training methods exist that allow
networks to learn by updating weights automatically (explored next week)
Expanding to More Inputs
artificial neurons may have many more than two input connections
calculation performed is the same: multiply each input by the weight of the connection, and find the sum of all of these products
notation can become unwieldy:• sum = x1·w1 + x2·w2 + x3·w3 + … + x100·w100
Some Mathematical Notation
Most references (e.g., Plunkett & Elman text) use mathematical summation notation
Sums of large numbers of terms are represented by the symbol Sigma ()– previous sum is denoted as:
100
xk·wk
k = 1
Summation Notation Basics
Terms are described once, generally Index variable shows range of possible
values Example:
5
k / (k - 1) = 3/2 + 4/3 + 5/4
k = 3
Summation Notation Example
Write the following sum using Sigma notation:3·x0 + 4·x1 + 5·x2 + 6·x3 + 7·x4 + 8·x5
+ 9·x6 + 10·x7
Answer: 7
(k + 3) ·xk
k = 0
Vector Notation
The most compact way to specify values for inputs and weights when we have many connections
The ORDER in which values are specified is important
Example: if w1 = 3.5, w2 = 1.74, and w3 = 18.2, we say that the weight vector w = (3.5, 1.74, 18.2)
Vector Operations
Vector Addition: adding two vectors means adding the values from the same position in each vector– result is a new vector
Example: (9.2, 0, 17) + (1, 2, 3) = (10.2, 2, 20)
Vector Subtraction: subtract corresponding values
(9.2, 0, 17) - (1, 2, 3) = (8.2, -2, 14)
Vector Operations
Dot Product: mathematical name for what a perceptron does
x · m = x1·m1 + x2·m2 + x3·m3 + … + xlast·mlast
Result of a Dot Product is a single number
example: (9.2, 0, 17) · (1, 2, 3) = 9.2 + 0 + 51 = 60.2
Perceptron Networks and Vector Notation
CS/PY 231 Lab Presentation # 3 January 31, 2005 Mount Union College