cs 330 - artificial intelligencecaora/cs330/materials/fall2018/...• interesting talk with students...
Post on 25-May-2020
4 Views
Preview:
TRANSCRIPT
CS 330 - Artificial Intelligence - Introduction
Instructor: Renzhi Cao Computer Science Department
Pacific Lutheran University Fall 2018
1
• Office: MCLT 248 • Office hours: In class website (cs.plu.edu/330) • Office Phone: 535-7409 • Email: caora@plu.edu
About me
Renzhi Cao
• Data Science • Machine learning • Bioinformatics
Pictures from: https://www.google.com/search?q=cow&biw=1920&bih=911&source=lnms&tbm=isch&sa=X&ved=0ahUKEwiOt5zlierOAhUE02MKHVbwDY8Q_AUIBigB#imgrc=0dSVh7Vlup1KqM%3A
2
About invited speaker
3
• Technology Management• Business Analytics• Innovation Strategy• Project Management
Assistant Professor from department of business
Leong Chan
About invited speaker
4
Assistant Professor from department of Math
J. Nicola
About you
• Names • Where are you from? • Your major? • Hobbies? Movie? Song? Sports? Book? TV show?
What you did over your break? … • Special skills?
6
What is this course?
What is this course about?
7
What is this course?
8
Group discussion:
• Think an example of AI applications that you could thought of, and share with your neighbors.
• Think one scenario that AI may not work or you don’t want it work? Share with your neighbors.
• Your new understanding of AI?
What is this course?
9
Group discussion:
• Big changes compared to last year: python, math background.
What is this course?
How do you solve a problem as a computer programmer?
For example, I have seen the following: 1 + 2 = 3 2 + 3 = 5 5 + 4 = 9 …… Q: 4 + 5 = ? 5 + 6 = ?
10
What is this course?
Machine learning VS traditional programming?
11
Traditional Programming
Data
Program
Machine
Output
1 + 2 = 3
Input two numbers: 1 2
What it cannot solve
Rules: Shape? Color? Eye? Mouth? …….
What is this course?
14
Data
output
Machine New Program
3
Input two numbers: 1 2
2 + 3 = ?
What is this course?
In this course, we are going to learn several machine learning techniques in AI, and use it to solve problems in different fields.
15
Syllabus
Attendance
Attendance • Expected to attend every class • YOU are responsible for missed materials
Classroom Conduct • Come to class on time • Turn off electronic devices • Refrain from private conversations (voice or electronic) • Refrain from activities unrelated to current tasks in class • Treat others with respect and dignity
18
Text book and meeting times
Books: • Machine Learning. By Tom M. Mitchell, PUBLISHED BY MIT PRESS, 1997.
• Introduction to machine learning. BY ALPAYDIN, ETHEM, PUBLISHED BY MIT PRESS, 2009
• Deep Learning. BY GOODFELLOW, IAN, PUBLISHED BY MIT PRESS, 2016
Time: • Tuesday, Thursday 9:55-11:40, MCLT #210 (Dr. Cao)
Course website
19
Course Website:
• https://www.cs.plu.edu/~caora/cs330/
• https://cs.plu.edu/330
Course Goals
20
Course goals: • Understanding machine learning concept • Developing problem solving skills • Applying machine learning techniques to solve problems in different fields • Having fun on machine learning techniques and developing skills that will
allow you to learn on your own!
Learning outcomes
21
Learning outcomes: • An ability to apply mathematics and the scientific method to solving computing
problems. • An ability to critically analyze a problem and to design, implement, and evaluate
a computing solution that meets requirements. • An ability to work effectively in small groups on medium scale computing
projects. • An ability to use oral and written communication effectively. • A recognition of the need to engage in life long learning. • An ability to understand the social and ethical implications of working as a
professional in the field of computer science. • An ability to use current tools and methodologies in computing practice.
Prerequisites
22
The official prerequisite for this course is Data Structure in CS 270. Students in Data Science minor could take DS 233 as prerequisite. Some programming experience is preferred, and math background is plus: • Linear algebra: vector/matrix manipulations, properties • Calculus: partial derivatives • Probability: common distributions; Bayes Rule • Statistics: mean/median/mode; maximum likelihood
Course Grade
23
Course participation - 10% • In-class exercises or assignments • Interactions in or out of class • Attendance • Attitude
Course Grade
24
Literatures will be provided by Professor Cao, but you can also find your own literature as long as you send it to Professor Cao before presentation for approval. It is group work, and each group needs to select a topic from a list.
Each group should email me if you decide to present a literature and topic as soon as possible. Ideally, each literature and topic should be presented by one group, and 10% deduction may be applied to other groups to present the same topic and literature.
The reports of literature review would be summary of literatures on the selected topic as a report, and slides of group presentation.
Literature Review - 10% • Around one literature review during the semester • Including report and presentation
Course Grade
25
The literature review will be around 25 mins (20 mins presentation and 5 mins Q&A) for each group, and will be evaluated by the following factor:
• Clearly present the background and its significance or motivation? (3 points)• Slides is clear and easy to follow. (3 points)• Present smoothly. You may need to practice ahead. (3 points)• The time. Not too early or too late. (3 points)• The literature itself. Some are easy to present, some are difficult. (3 points)• Extra points is available if you are actively interacting with other group’s
presentation.
Literature Review - 10% • Around one literature review during the semester • Including report and presentation
Course Grade
26
Quiz - 20% • Several quizzes during the semester
Course Grade
27
Labs - 25% • Around 4 labs during the semester
Course Grade
28
Final exam - 15% • One final written exam
Course Grade
29
Course Project - 20% • Mid-term proposal presentation (5%) • Final project result presentation (5%) • Reports and codes (10%) • Extra points based on novelty and performance of your
methods
Overall Score Grade100% -- 90% A / A-
90% -- 80% B+ / B / B-
80% -- 70% C+ / C / C-
70% -- 60% D+ / D / D-
60% -- 0% E
Policies of collaborations and assignments
Not allowed for assignments or in-class exercises.Allowed for project or literature review, but contributions of each person need to be included in report. Cite references and acknowledge others work.
If students begin working on a project as groups and cannot complete it together, at least one student must contact the instructor to request a partnership dissolution.
Assignments or in-class exercises must be submitted before the due date. A late penalty of 10% per day will be assessed after due date, except that you have a strong reason - an emergency, illness, or absence due to a university sanctioned activity such as a sporting event or music performance.
Important time
• Last day to add a class without a fee: Sep. 11th• Last day to drop a class without a fee: Sep. 17th• Last day to withdraw: Nov. 26th
Introduction
33
When machine learning starts?
Inventors have long dreamed of creating machines that can learn. Desires date back to at least the time of ancient Greece.
Inventors Pygmalion and statue he carved - Galatea
Talos - a giant automaton made of bronze to protect Europa in Crete from pirates and invaders, by inventor Daedalus
Inventor Hephaestus and the first human woman created by him - Pandora
Introduction
When machine learning starts?
People wonder whether such machines may become intelligent. Today, Artificial intelligence (AI) is a thriving field with many practical applications and active research topics.
Machine learning in early days
Used to solve problems that are intellectually difficult for human beings but relatively straightforward for computers, based on a list of formal and mathematical rules.
The true challenge for machine learning is to solve tasks that are easy for people but difficult for machine to do. Recognizing spoken words, faces in images.
Machine learning in now days
Introduction
35
Why machine learning?
• My experience at Silicon Valley. (13 w)
• It’s everywhere. You may not want to know how to make a car, but it’s always good if you know something about it.
• A dream that one day the machine is as intelligent as human.
Introduction
36
Why machine learning?More importantly, the traditional program can not solve some problems, such as recognizing a three-dimensional object from a novel view in new lighting conditions in a cluttered scene.
• What to write?
• Even you know what to write, it’s too complicate.
Introduction
37
Why machine learning?
What rule to decide your credit card transaction is fraudulent?
• There may be simple rules, but we need to combine a lot of weak rules
• Rules will be changed, so your method needs to be updated all the time.
Introduction
38
Machine learning approach
Instead of writing program with all different rules, we collect examples that specify the correct output for a given input.
• The machine created program may look very different from typical program. It may contains millions of numbers
• If we do it right, the machine created program works well for new cases, and also the ones we trained on it
• If the data changes, the program can also be changed by training on new data
A machine learning algorithm takes the examples and produces program that does the job
Massive amounts of computation are now cheaper than paying someone to write-specific program.
Spam Filtering Web SearchPostal Mail
Routing
Fraud DetectionMovie
RecommendationsVehicle Driver
Assistance
Web Advertisements
Social NetworksSpeech
Recognition
Machine Learning in Our Daily Lives
Introduction
Google search demo
Introduction
Google translation demo
Introduction
42
What is this course?
43
Introduction
44
Introduction
45
Self-driving car
What is this course?
46
Introduction
47
Amazon recommendation
Art
Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
Art
Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
Art
Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. "Perceptual losses for real-time style transfer and super-resolution." arXiv preprint arXiv:1603.08155 (2016).
The Muse, Pablo Picasso, 1935
Computational biology>T0759 HR9083A, Human, 109 residues MGHHHHHHSHMVVIHPDPGRELSPEEAHRAGLIDWNMFVKLRSQECDWEEISVKGPNGES SVIHDRKSGKKFSIEEALQSGRLTPAHYDRYVNKDMSIQELAVLVSGQK
More interesting AI
• http://simon.cs.plu.edu/MLFun//index.php
Introduction
53
Machine VS Human
• IBM deep blue chess-playing system defeated world champion Garry Kasparov in 1997.
March 2016
AlphaGo 4 – Lee Sedol 1
https://www.britgo.org/intro/intro2.html
The rules• Starts with an empty board. • Players take turns to place one stone on vacant point • Capture your opponent’s stones by completely surrounding them • Goal: Use your stones to form territories by surrounding vacant
areas of the board
Baidu’s AI boss, Andrew Ng, pictured left with the host of ‘Super Brain’ and Baidu’s robot, XiaoduJan. 2017
https://www.theguardian.com/technology/2017/jan/30/libratus-poker-artificial-intelligence-professional-human-players-competition
Jan. 2017
SWARM AI CORRECTLY PREDICTED THE OUTCOME OF SUPER BOWL LI, RIGHT DOWN TO THE FINAL SCORE
http://www.digitaltrends.com/cool-tech/swarm-artificial-intelligence-super-bowl-patriots/
The New England Patriots’ win over the Atlanta Falcons was nothing short of amazing. The Pats rallied back from a 25-point deficit to tie the game in the final minutes of regulation and secured the win with a decisive touchdown drive in overtime. Swarm AI (Combines swarming algorithms with human input) accurately predicted the outcome of the game, right down to the 34-28 win by the Patriots.
February 6, 2017
Introduction
62
Machine learning VS AI• Interesting talk with students
• Artificial Intelligence is the broader concept of machines being able to carry out tasks in a way that we would consider “smart”.
• Machine Learning is a current application of AI based around the idea that we should really just be able to give machines access to data and let them learn for themselves.
http://www.forbes.com/sites/bernardmarr/2016/12/06/what-is-the-difference-between-artificial-intelligence-and-machine-learning/#2d597284687c
CHAPTER 1. INTRODUCTION
AI
Machine learning
Representation learning
Deep learning
Example:Knowledge
bases
Example:Logistic
regression
Example:Shallow
autoencodersExample:MLPs
Figure 1.4: A Venn diagram showing how deep learning is a kind of representation learning,which is in turn a kind of machine learning, which is used for many but not all approachesto AI. Each section of the Venn diagram includes an example of an AI technology.
9
The MNIST Dataset
CHAPTER 1. INTRODUCTION
Figure 1.9: Example inputs from the MNIST dataset. The “NIST” stands for NationalInstitute of Standards and Technology, the agency that originally collected this data.The “M” stands for “modified,” since the data has been preprocessed for easier use withmachine learning algorithms. The MNIST dataset consists of scans of handwritten digitsand associated labels describing which digit 0–9 is contained in each image. This simpleclassification problem is one of the simplest and most widely used tests in deep learningresearch. It remains popular despite being quite easy for modern techniques to solve.Geoffrey Hinton has described it as “the drosophila of machine learning,” meaning thatit allows machine learning researchers to study their algorithms in controlled laboratoryconditions, much as biologists often study fruit flies.
22
Goodfellow, 2016
Introduction
Historical Trends: Growing Datasets
CHAPTER 1. INTRODUCTION
1900 1950 1985 2000 2015
Year
10
0
10
1
10
2
10
3
10
4
10
5
10
6
10
7
10
8
10
9
Dataset
size
(num
ber
exam
ples)
Iris
MNIST
Public SVHN
ImageNet
CIFAR-10
ImageNet10k
ILSVRC 2014
Sports-1M
Rotated T vs. C
T vs. G vs. F
Criminals
Canadian Hansard
WMT
Figure 1.8: Dataset sizes have increased greatly over time. In the early 1900s, statisticiansstudied datasets using hundreds or thousands of manually compiled measurements (Garson,1900; Gosset, 1908; Anderson, 1935; Fisher, 1936). In the 1950s through 1980s, the pioneersof biologically inspired machine learning often worked with small, synthetic datasets, suchas low-resolution bitmaps of letters, that were designed to incur low computational cost anddemonstrate that neural networks were able to learn specific kinds of functions (Widrowand Hoff, 1960; Rumelhart et al., 1986b). In the 1980s and 1990s, machine learningbecame more statistical in nature and began to leverage larger datasets containing tensof thousands of examples such as the MNIST dataset (shown in figure 1.9) of scansof handwritten numbers (LeCun et al., 1998b). In the first decade of the 2000s, moresophisticated datasets of this same size, such as the CIFAR-10 dataset (Krizhevsky andHinton, 2009) continued to be produced. Toward the end of that decade and throughoutthe first half of the 2010s, significantly larger datasets, containing hundreds of thousandsto tens of millions of examples, completely changed what was possible with deep learning.These datasets included the public Street View House Numbers dataset (Netzer et al.,2011), various versions of the ImageNet dataset (Deng et al., 2009, 2010a; Russakovskyet al., 2014a), and the Sports-1M dataset (Karpathy et al., 2014). At the top of thegraph, we see that datasets of translated sentences, such as IBM’s dataset constructedfrom the Canadian Hansard (Brown et al., 1990) and the WMT 2014 English to Frenchdataset (Schwenk, 2014) are typically far ahead of other dataset sizes.
21
Figure 1.8Goodfellow, 2016
66
Biological neural network size from Wikipedia (2015).
Introduction
Number of Neurons
Figure 1.11Goodfellow, 2016
CHAPTER 1. INTRODUCTION
1950 1985 2000 2015 2056
Year
10
�2
10
�1
10
0
10
1
10
2
10
3
10
4
10
5
10
6
10
7
10
8
10
9
10
10
10
11
Num
ber
ofneurons
(logarithm
ic
scale)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Sponge
Roundworm
Leech
Ant
Bee
Frog
Octopus
Human
Figure 1.11: Since the introduction of hidden units, artificial neural networks have doubledin size roughly every 2.4 years. Biological neural network sizes from Wikipedia (2015).
1. Perceptron (Rosenblatt, 1958, 1962)
2. Adaptive linear element (Widrow and Hoff, 1960)
3. Neocognitron (Fukushima, 1980)
4. Early back-propagation network (Rumelhart et al., 1986b)
5. Recurrent neural network for speech recognition (Robinson and Fallside, 1991)
6. Multilayer perceptron for speech recognition (Bengio et al., 1991)
7. Mean field sigmoid belief network (Saul et al., 1996)
8. LeNet-5 (LeCun et al., 1998b)
9. Echo state network (Jaeger and Haas, 2004)
10. Deep belief network (Hinton et al., 2006)
11. GPU-accelerated convolutional network (Chellapilla et al., 2006)
12. Deep Boltzmann machine (Salakhutdinov and Hinton, 2009a)
13. GPU-accelerated deep belief network (Raina et al., 2009)
14. Unsupervised convolutional network (Jarrett et al., 2009)
15. GPU-accelerated multilayer perceptron (Ciresan et al., 2010)
16. OMP-1 network (Coates and Ng, 2011)
17. Distributed autoencoder (Le et al., 2012)
18. Multi-GPU convolutional network (Krizhevsky et al., 2012)
19. COTS HPC unsupervised convolutional network (Coates et al., 2013)
20. GoogLeNet (Szegedy et al., 2014a)
27
Introduction
Solving Object Recognition
Figure 1.12Goodfellow, 2016
CHAPTER 1. INTRODUCTION
2010 2011 2012 2013 2014 2015
Year
0.00
0.05
0.10
0.15
0.20
0.25
0.30
ILSV
RC
classification
error
rate
Figure 1.12: Since deep networks reached the scale necessary to compete in the ImageNetLarge Scale Visual Recognition Challenge, they have consistently won the competitionevery year, and yielded lower and lower error rates each time. Data from Russakovskyet al. (2014b) and He et al. (2015).
28
Introduction
What is machine learning?
Introduction
Machine learning definitionArthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed.
Introduction
Machine learning definition
Tom Mitchell provides a more modern definition: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
Example: playing checkers. E = the experience of playing many games of checkers T = the task of playing checkers. P = the probability that the program will win the next game.
Introduction
Machine learning types
• Supervised learning • Unsupervised learning • Reinforcement Learning
Introduction
Supervised learning • Learn to predict an output when given an input vector
UnSupervised learning • Learn to select an action to minimize payoff
Reinforcement Learning • Discover a good internal representation of the input
Introduction
Supervised learning
• Each training case consists of an input vector x and a target output t.
• Regression: the target output is a real number or a whole vector of real numbers.
• Classification: the output is a class label.
Introduction
Supervised learning
We start by choosing a model-class: y = f(x,W) • A model-class f is a way of using some numerical parameters
W to map each input vector x into a predicted output y.
Learning usually means adjusting the parameters to reduce the discrepancy between the target output t on each training case and the actual output y, which produced by the model. • For regression, we usually use the following as sensible measure of
discrepancy: (y-t)2/2.
• For classification, there are other measures that are generally more sensible.
850
Supervised Learning
Given “Right answers”
Regression:
output is a continuous value
Classification
Output is discrete value (0 or 1, or 2, and etc.)
2.5
Breast
Introduction
Unsupervised learning
For about 40 years, unsupervised learning was largely ignored by the machine learning community • Some widely used definition of machine learning
actually excluded it • Many researchers thought that clustering is the only
form of unsupervised learning.
It’s hard to define what’s the goal of unsupervised learning.
x1
x2
Unsupervised Learning
No “Right answers”
https://news.google.com/
http://www.zlti.com/subdomains/analytics/Technology.html
https://bmcstructbiol.biomedcentral.com/articles/10.1186/1472-6807-14-13
Introduction
Reinforcement learning
In reinforcement learning, the output is an action or sequence of actions and the only supervisory signal is an occasional scalar reward. • The goal in selecting each section is to maximize the
expected sum of the future rewards. • We usually use a discount factor for delayed rewards so
that we didn’t have to look too far into the future
Reinforcement learning is difficult: • The rewards are typically delayed so it’s hard to know where we
went wrong (or right). • A scalar reward does not supply much information.
Introduction
• Exercises • Check course website
Useful resources
85
There will be several useful resources:
• Sakai for announcement and assignments • Course website : https://cs.plu.edu/330 • GitLab: full featured GitHub style system, but it is self-hosted on PLU servers
https://gitlab.cs.plu.edu
Useful resources
86
87
Resources
Before you leave today… - apply for a curly account
https://www.cs.plu.edu/hub/accounts/requests/newFinish survey about your background (available on course website).
Register and change your password on course website.
88
Resources
Create an Account…– Open the Firefox browser– Go to https://www.cs.plu.edu/hub/– Click on Request link– Review PLU Policies– Click on I agree link
89
Resources
Account for the first few days (active through Sep. 21, 2018):
user name: firstday password: fall2018
Topics (Theme of AI in Business)• AI in Finance • AI in Marketing • AI in HR • AI in Cyber security like Cryptocurrency • AI in Healthcare (Bioinformatics) • AI in supply-chain and logistics (Amazon) • AI in Autonomous car • Other AI applications in Business
Tasks in summary
2. Think about topics that you are interested and form group, Proposing interesting machine learning topics.
1. Explore gitlab / github
3. Read materials in course website
4. Labs, and your computer availability (Python programming)
5. Request account before first day account expired
6. Finish survey, check out class website and Sakai regularly: https://www.cs.plu.edu/~caora/cs330/
QuestionsDiscussion about topics
top related