machine learning: a brief history 1cmdavid barbermpd37/teaching/2014/ml_tutorials/2014... ·...

Machine Learning: A brief history

David Barber

Outline

1 History of the AI dream

2 How do brains work?

3 ConnectionismImage ProcessingLearning to ‘talk’

4 Fantasy MachinesFinding the BurglarSpelling correctionSpeech recognitionMedical diagnosis

5 Physical Interaction and Commonsense

6 Current Research in Machine Learning

7 Outlook

Engineering Intelligence







7 Outlook

Intelligent Machinery

1948 Turing and Champernowne ‘paper and pencil’ chess.

Intelligent Machinery

1951 Prinz mate-in-two moves chess machine.1952 Strachey programs first computer draughts algorithm.

Learning Machines

1951 Oettinger makes first program that ‘learns’.1955 Samuel adds ‘learning’ to his draughts algorithm.

Logical Intelligence

1968 Risch’s algorithm for integration in calculus

1972 Prolog for general logical reasoning

1997 Deep Blue defeats Kasparov

Other forms of intelligence

But is this getting us to where we’d like to be?Selfridge-Shannon film clip

Speech Recognition

Visual Processing

Natural Language modelling

Planning and decision in uncertain environments

Perhaps a different approach would be useful.








7 Outlook

Astonishing Hypothesis: Crick

“A person’s mental activities are entirely due tothe behaviour of nerve cells and the moleculesthat make them up and influence them.”

Neurons

Visual Pathway

Information Processing in Brains

Neurons

Real W

orl

d

Layer 1 Layer 2 High−level

Concepts

Feature

Hierarchical; Modular; Binary; Parallel; Noisy








7 Outlook

McCulloch and Pitts Network (1943)

Artificial Neuron (Perceptron)

weight 7

output neuron

neuron 1neuron 2neuron 3neuron 4

neuron 7neuron 6neuron 5

inputs

weight 1

Training an artificial neural network

Want to generalise to new images with high accuracy.

Artificial Network

1957 Rosenblatt’s perceptron

perceptron film clip

Connectionism

1960 Realised a perceptron can only solve simple tasks.

1970 Decline in interest.

1980 New computing power made training multilayernetworks feasible.

outputinputs








7 Outlook

Digit Recognition using a Neural Network

Computer makes error less than 1/1000.

Digit recognition using a neural network

Feature neurons in the first layer

Feature neurons in a real brain








7 Outlook

NETtalk: Sejnowski and Rosenberg (1986)

NETtalk clip

Limitations of forward reasoning

World Representation

Recognising patterns (perceptron style) is only oneform of intelligence.

Solving chess problems is another and requirescomplex reasoning using some form of internal model.

The world is noisy and information may be conflicting.

Recognised that new approaches are required.








7 Outlook

Limitations of forward reasoning

World Representation

Models help us to fantasise about the world.

Models

Models

Re

al

Wo

rld

High−level

Concepts








7 Outlook

Burglar Problem

Creaks and Bumps

Creak Bump

Burglar Model

pos1 pos2 pos3 pos4

snd1 snd2 snd3 snd4

pos - position in kitchensnd – sound

Finding the Burglar

creak! creak!

bump!

creak!

bump! bump!

creak!

bump! bump! bump!

creak!

bump!








7 Outlook

Stubby Fingers

Stubby Fingers

int1 int2 int3 int4

hit1 hit2 hit3 hit4

int - intended keyhit – hit key

Stubby Fingers: errors

a b c d e f g h i j k l m n o p q r s t u v w x y z

abcdefghijkl

mnopqrstuvwxyz

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Stubby Fingers: language

a b c d e f g h i j k l m n o p q r s t u v w x y z

abcdefghijkl

mnopqrstuvwxyz 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Stubby Fingers

Given the typed sequence kezrninh what is the mostlikely word that this corresponds to?

Listing the 200 most likely hidden sequences

Discard those that are not in a standard Englishdictionary

Take the most likely proper English word as theintended typed word

. . . and the answer is . . .

learningWhat about cwsykcak

?

Stubby Fingers

Given the typed sequence kezrninh what is the mostlikely word that this corresponds to?

Listing the 200 most likely hidden sequences

Discard those that are not in a standard Englishdictionary

Take the most likely proper English word as theintended typed word

. . . and the answer is . . . learningWhat about cwsykcak ?








7 Outlook

Speech Recognition: raw signal

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

Time

‘neural’ representation

10 20 30 40 50 60 70 80

5

10

15

20

25

Speech Recognition

pho1 pho2 pho3 pho4

aud1 aud2 aud3 aud4

pho: phoneme (letter)aud: audio signal (neural representation)








7 Outlook

Medical Diagnosis

tumour flu meningitis

headache fever appetite x-ray

Combine known medical knowledge with patient specificinformation.








7 Outlook

Robots

robot film clip big dog clip big dog beta clip

Commonsense Reasoning

Recognising an object based on its features is useful(but limited).

Understanding what ‘fetch me a teacup’ means ishard.

Something that can contain hot water that a humancan drink comfortably from.

“Mary saw the dog in the shop window. Shewanted it.”

Machines ultimately need to understand more aboutwhat it means to be human.








7 Outlook

ML Research Communities

Automatic car navigation.

Monitoring behaviour of plants/machines, surveillance.

Search:‘What is the tallest building in the world?’versus‘Does the same person appear in both these videos?’

Translation.

Scientific reasoning.

Medical diagnosis.

Speech recognition

Natural Language Processing

Face Recognition

Robot Control

Game Playing

Object Recognition

Bio-informatics

Personalised Medicine

Prediction

stock marketconsumer demand e.g. newspapersengine failure








7 Outlook

We’re only just beginning...

AI is hard!

Machines can do amazing things – beat us at chess,navigate autonomously, recognise our speech.

Tremendous progress has been made in theseextremely difficult problems.

Need to develop algorithms that work physically inparallel, like brains do and harness true power ofparallel computing.

Need continued progress in processing of complexsignals, modelling of environment, understandingwhat it means to be human.

Need continued insights from biology about hownatural organisms solve some of these complex tasks.

Thankyou!

www.cs.ucl.ac.uk/staff/D.Barber

machine learning: a brief history 1cmdavid barbermpd37/teaching/2014/ml_tutorials/2014... ·...

Documents