computers - using your brains jim austin professor of neural computation
Post on 15-Jan-2016
225 views
TRANSCRIPT
![Page 1: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/1.jpg)
Computers - Using your Brains
Jim Austin
Professor of Neural Computation
![Page 2: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/2.jpg)
![Page 3: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/3.jpg)
![Page 4: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/4.jpg)
Pentium III
![Page 5: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/5.jpg)
So how complex is it ?
– 1012 neurons … 1,000,000,000,000–1000 connections between neurons.
– One brain can hold ...
1,000,000,000,000,000 numbers!
![Page 6: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/6.jpg)
What do 1012 neurons look like ?
• 1600 times Population of the world (6,100,000,000)
• 78,125 times the complexity of the Pentium III
• Equal to the number of stars in our galaxy
4 Meters
4 Meters
4 Meters
![Page 7: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/7.jpg)
The good and the bad
What are computers bad at ?Being reliableFinding information - knowledgeDoing very complex things - recognizing imagesLearning to do the job them selves!
What are computers good at ?Adding up fastStoring data - numbers and factsPushing data around
![Page 8: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/8.jpg)
Why are computers so restricted ?
ACE
![Page 9: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/9.jpg)
Leo - for stock control
![Page 10: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/10.jpg)
Colossus - for breaking codes
![Page 11: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/11.jpg)
Pegusus - for scientific work
![Page 12: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/12.jpg)
Neurons verses Gates
NAND Gate
Input 1Input 2
Output
Boolean Logic - both inputs OK, output not OK
![Page 13: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/13.jpg)
Gates - NAND
Input 1 Input 2 Output
==
=
ALL inputs to be OK for output to be NOT OK
![Page 14: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/14.jpg)
![Page 15: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/15.jpg)
Evolution ?
Should have picked a NAND gate for the brain...
![Page 16: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/16.jpg)
+
“Weights”
Inputs OutputA
B
Threshold logic - threshold 1 - one or more inputs OK output OK
Output = threshold (input A x weight A + input B x weight B)
Neuron
![Page 17: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/17.jpg)
Neuron
==
=
At least three OK’s for output to be OK
At least one OK for output to be OK
![Page 18: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/18.jpg)
Can also alter connections/importance of inputs
using the weights on the inputs
+3.5
1
0
1
1
0.5
1
1
Weights
![Page 19: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/19.jpg)
Why did this difference develop ?
• “The analysis of the operation of a machine using two indication elements and signals can be conveniently be expressed in terms of a diagrammatic notation introduced, in this context, by Von Neuman and extended by Turing. This was adopted from a notation used by Pits and McCulloch as a possible way of analyzing the operation of the nervous system,…” Calculating Instruments & Machines, D Hartree, 1950, Cambridge University Press.
• Probably dropped due to the development of the silicon chip– simpler to build Boolean logic gates rather than neuron
units.
![Page 20: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/20.jpg)
Functional elements.
nz
k inputs
Threshold n gatek n
1z Excitation, “OR”
2z
Excitation, “AND”
![Page 21: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/21.jpg)
ICT Orion Computer• Used ‘Neuron’ logic - 1962
![Page 22: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/22.jpg)
![Page 23: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/23.jpg)
Learning !
Learning at neuron level =
Adjustment of which inputs are important
Conventional computers have no implicit learning ability
![Page 24: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/24.jpg)
Spot the difference
![Page 25: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/25.jpg)
![Page 26: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/26.jpg)
Happy
Hungry
+
+
Threshold = 2
![Page 27: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/27.jpg)
+
+ Hungry
Happy
![Page 28: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/28.jpg)
+
+ hungry
Happy
![Page 29: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/29.jpg)
Can we build useful systems with neurons ?
Better tolerance to failureParallelism/use of threshold logic/distributed memory
Faster operationMassive parallelism
Better access to uncertain informationThreshold logic/neurons
Where the inputs are uncertainThreshold logic/neurons.
Where we want low powerAsynchronous systems
AdaptabilityUse of weights and learning methods.
![Page 30: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/30.jpg)
So what have we done with these ?
Cortex-1
28 Processor cards, each holding 128 hardware neurons.
Each with 1,000,000,000 weights.
16MHz.
PCI based card.
![Page 31: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/31.jpg)
Complete Machine:
400,000,000 neuron evaluations per second28,000 inputs30 bits set on input1,000,000 neurons.
![Page 32: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/32.jpg)
Cortex-1 node
5,120,000,000 neuron weights, 640 neurons.
![Page 33: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/33.jpg)
Recognising Addresses for the Post Office
![Page 34: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/34.jpg)
Recognising trademarks
![Page 35: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/35.jpg)
Text search engines
• Tolerant to spelling errors.
• Finds similar words to those supplied, for example chair, seat, bench.
• Learns these similarities automatically from text.
• Uses neural engine for document storage.
• Estimated 400,000,000 documents searched per second.
![Page 36: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/36.jpg)
Molecular Databases• One of few systems that deal with the full 3D molecule
![Page 37: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/37.jpg)
![Page 38: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/38.jpg)
Query
Good matches
Bad Match
![Page 39: Computers - Using your Brains Jim Austin Professor of Neural Computation](https://reader035.vdocuments.us/reader035/viewer/2022062221/56649d375503460f94a0f7b6/html5/thumbnails/39.jpg)
(It’s Brains from Thunderbirds !)
Aaron Turner
Mick Turner
Vicky Hodge
Julian Young
Anthony Moulds
Zyg Ulanowski
Ken Lees
Michael Weeks
Sujeewa Alwis
John Kennedy
David Lomas
and many others ….
Thanks...