alex acm sc machine learning day [materials] | machine learning& artificial intelligence ... our...

7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed

1/69

Artificial Intelligence

Our Ultimate Dream

Mohammad Abdelaziz Gowayyed

May 2013


2/69

Agenda

Introduction

Example: How to build a machine to play

chess?

Artificial Intelligence Subfields

Feature Extraction for Classification

Artificial Neural Networks

Future of Artificial Intelligence


3/69

What is Artificial Intelligence?

The study of computer systems that attempt

to model and apply the intelligence of the

human mind


4/69

Artificial Intelligence Directions

Build Systems that:

act like humans

act rationally

think like humans

think rationally


5/69

Act like human: The Turing Test

Turing Test:

If a man had a conversation with a machine and

he couldnt tell, whether it was a machine or a

human, this machine passes the Turing Test

It means that the machines perfectly acts like a

human

Does it need to think like a human to act like ahuman?


6/69

Think like human: Cognitive Science

We must somehow determine first how

human think, do we know?

Introspection

Psychological experiments

Cognitive Science: construct precise and

testable theories of the workings of humanminds through computer models and

psychological experiments.


7/69

Think rationally: Laws of Thought

Formalizing logic

Main emphasis is on inference

Example: Socrates is a man; all men are

mortal; therefore Socrates is mortal.

By 1965, programs existed that could, givenenough time and memory, take a description

of a problem in logical notation and find thesolution to the problem, if one exists.

Problems?


8/69

Act rationally: The rational agent

Rational behavior: doing the right thing

The right thing: that which is expected to

maximize goal achievement, given the

available information

Take decisions

Reinforcement Learning: learn fromenvironment feedback based on my choices


9/69

Is there a benchmark for humanintelligence?

May be Chess?


10/69

How to program a machine to play

Chess?


11/69

How to program a machine to play

Tic Tac Toe?


12/69

Easy?


13/69

What is the difference betweenChess and Tic Tac Toe?


14/69

The general idea is to build a treerepresenting different possible paths,

each path represents sequence of

choices from you and your opponent.The tree will help the computer

choose

That is called aGame Tree


15/69


16/69

Min-Max Algorithm

Two agents are playing

Zero-Sum Game

Any loss to you is an equal win for your opponent

Assume that you are playing with a rational

player; he will try to maximize his profit

At each step, you want to maximize your profitand assume that the opponent will minimize it


17/69


18/69

Min-Max Algorithm

Is it feasible to create the whole tree? Is it finite?

What about the depth?

Will we be forced to choose random moves?

How to evaluate each state? Heuristic function (e.g. number of safe pieces)

Different algorithms for traversing the tree Depth first

Breadth First A*

Do we need to visit the whole tree? Alpha-Beta pruning


19/69

Heuristic Function


20/69

Can we say that the computer is

now intelligent because it candefeat human in chess?


21/69

There are still a lot of tasks that

human does easily while it is very

hard for machines

Can you name a few?


22/69

Chinese Room


23/69

Chinese Room


24/69

Machine vs. Human

Machine

Easy Tasks

Multiply 4878 * 1254

Play chess Hard Tasks

Understand concepts

Recognize faces

Recognize voices

Human

Easy Tasks

Recognize faces

Recognize voices Understand concepts

Hard Tasks

Multiply 4878 * 1254

Play chess


25/69

Some Artificial Intelligence Subfields

Machine Learning

Let machines learn rules (e.g. to classify or tochoose between alternatives Reinforcement

Learning) Computer Vision

Teach machines how to recognize visual world

Natural Language Processing

Teach machines how to recognize humanlanguages

Speech Recognition

Teach machines how to recognize human voices


26/69

Some Computer Vision Tasks


27/69

Face Detection


28/69

Image Classification


29/69

Human Action Recognition


30/69


31/69

Feature Representation


32/69



33/69

How is computer perception done?


34/69



35/69

Computer Vision Features


36/69

Audio Features


37/69

NLP Features


38/69

Sensor Representation in Human Brain


39/69

Can we mimic what already happens in

human brains?

The human brain consists of a network of

neurons.

Human brains learns to recognize images by

the same method as it learns to recognize

voices.

Scientists still dont know how the brain

works.

Scientists only know some concepts that brain

applies (e.g. sparse and general

representations)


40/69

Human Nervous System

Around 85,000,000,000neurons each one

connected to around 1000 others


41/69

Number of neurons in elephant brains

11,000,000,000 neurons


42/69

Number of neurons in horses brains

1,200,000,000 neurons


43/69

Neuron


44/69

Neuron


45/69

Number of neurons in human brains

From 19,000,000,000 to

23,000,000,000 neurons


46/69

Artificial Neural Networks

We know that human brains consists of hugeneural network, lets study simpler artificialneural networks.

And some day, we may be able to createartificial neural network that works the sameas the real neural network works.

Some people thought that it will never work!They had their arguments.

Thats why some researchers dont like neural

networks.


47/69

Neural Networks


48/69

Neural Networks


49/69

Backpropagation


50/69

Backpropagation


51/69

How to build better Neural Networks? Increase Layers?

How would we train them?

The network with more than 1 hidden layer is

called a deep network

Training a deep network is not easy using

Backpropagation.


52/69

The birth of Deep Learning Until 2006, Support

Vector Machines always

outperformed neural

networks on benchmarkclassification tasks.

At 2006, Hinton started

the new revolution on

machine learning: Deep

Learning


53/69

The birth of Deep Learning At 2006, Hinton proposed a greedy layer-wise

algorithm for training a deep belief network.

That was the first successful learning

algorithm for a deep network. Achieved best accuracy on MNIST hand

written digits dataset.

Restricted Boltzmann Machines. The network works also as a generative model

to generate data.


54/69

Deep Belief Network


55/69

Demo


56/69

Classical Approach Train a classifier using labeled data; then test

the classification accuracy on another testing

data.

Row data can be: images, videos, audio,

natural language

We start with extracting appropriate features.

Hand-crafted feature extraction will differ

according to the nature of the data.


57/69

Deep Learning Approach Pre-train the network using non-labeled data-

available.

Fine-tune the weights using some labeled

data. Also called: Unsupervised Feature Learning

Deep Learning Models:

Deep Belief Networks Stacked Autoencoders

Convolutional Neural Networks


58/69

Sparse Autoencoders


59/69

What each neuron learnt?


60/69

What each neuron learnt?


61/69

Motivating Results Unsupervised feature extraction

outperformed hand-crafted features in set of

popular machine recognition tasks:

Human Activity Recognition

Handwritten digits Recognition

Audio classification

ImageNet classification


62/69

Biological Mapping Interesting paper at 2005: Invariant visual

representation by single neurons in the

human brain

They showed that some neurons respond with

high selectivity to, for instance, images, sound

and name of the actress Halle Berry!

How Many Computers to Identify a


63/69


Cat? 16,000

Andrew NG, with Google at 2012, trained a

very large network: on16,000 machines.

They trained the network on random

unlabeled videos from Youtube, without any

guidance.

They found that one neuron of the network

learnt to detect human faces and another onelearnt to detect cat faces!


64/69

Some neurons learnt human face!


65/69

Other neurons learnt cat faces!



66/69


Cat? 16,000


67/69

Future of Artificial Intelligence

With the increase of computational power,

every day, scientists are able to build more

complex neural networks

Challenges

Network topologies

Learning algorithms

Neuroscientists can lead an AI revolution if

they came closer to how the brain works


68/69

Acknowledgment

Some slides are taken from Andrew Ng deep

learning tutorial.

Images used here are for educational

purposes, any further use should return back

to the original copyright holders.


69/69

Thanks You

Questions?

alex acm sc machine learning day [materials] | machine learning& artificial intelligence ... our...

Documents