Lecture 0: Machine Learning

Tuo Zhao

Schools of ISYE and CSE, Georgia Tech

2017 Fall

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Questions

Course Logistics

Why Machine Learning?

What is a well-defined learning problem?

What questions should we ask about Machine Learning?

Tuo Zhao — Lecture 0: Machine Learning 2/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Machine Learning is Interdisciplinary

Tuo Zhao — Lecture 0: Machine Learning 3/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Pre-requisites

Math:

Calculus and Linear Algebra

Probability and Statistics

Basic Optimization

Coding:

MATLAB for coding HW (No Exception)

Plus:

Generalized Linear Models

Convex Optimization

Tuo Zhao — Lecture 0: Machine Learning 4/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Course Logistics

Teaching Assistants:

Shaojun Ma: Ph.D. Student in CEE

Yujia Xie: Ph.D. Student in CSE

Minshuo Chen: Ph.D. Student in ISYE

Haoming Jiang: Ph.D. Student in ISYE

Zhehui Chen: Ph.D. Student in ISYE

TBD

See http://www2.isye.gatech.edu/~tzhao80/others.html

Syllabus, Lecture Slides

Homework Assignments

Tuo Zhao — Lecture 0: Machine Learning 5/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Highlights of Course Logistics

Working Load:

Background Knowledge Test: 6%

4 Written HW: 24%

3 Coding HW: 18%

Exam-1: 26%

Exam-2: 26%

See https://piazza.com/class/j4ujbo0admd2in

Relase Important Announcements!

MUST Register!

You can post anonymously (to other students, but not me)

Tuo Zhao — Lecture 0: Machine Learning 6/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Distance Learning

Working Load:

5 Written HW: 60%

4 Coding HW: 40%

No Background Knowledge Test

No Mid-term Exam

Late Homework Policy for All Students:

No Late Homework Accepted!

Always due at noon on Friday

Tuo Zhao — Lecture 0: Machine Learning 7/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Knowledge Background Test Statistics

Top 10%: 30/40

Top 25%: 27/40

Top 50%: 20/40

Top 75%: 14/40

Maximum: 38

Suggestions – Go through the review materials carefully!

Tuo Zhao — Lecture 0: Machine Learning 8/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Remarks

Office hours for asking questions

Sep. 19 – A more difficult make-up exam (but will be curvedaccordingly)

No time for answering questions after class

You need to debug by yourself

Honor Code

Tuo Zhao — Lecture 0: Machine Learning 9/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

What to Cover?

Methodology and Algorithms of Machine Learning.

Some Theory for Ph.D. Students.

Some homework problems will be for Ph.D. students ONLY.

Different letter grades for each section.

Not About “Introduction to Machine Learning”

Not About “How to Apply Machine Learning to YourDomain”.

Not About “How to Use Software to Do Machine Learning”

Tuo Zhao — Lecture 0: Machine Learning 10/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Alternative

Easier: CS 4641 Machine Learning

Signal Processing: ECE 6254: Statistical Machine Learning

Learning Theory: CS 7545 Machine Learning Theory

More Foundation: CS 8803 Mathematical Foundations ofMachine Learning

Applications to Specific Domains: Computer Vision, NaturalLanguage Processing, etc.

Tuo Zhao — Lecture 0: Machine Learning 11/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Why Machine Learning?

Recent progress in algorithms and theories

Growing flood of massive data

Computational power is available

Budding industry

Tuo Zhao — Lecture 0: Machine Learning 12/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Why Machine Learning?

Three Niches for Machine Learning:

Data mining: using historical data to improve decisions

medical records → medical knowledge

Software applications we can’t program by hand

autonomous drivingspeech recognition

Self customizing programs

Newsreader that learns user interests

Tuo Zhao — Lecture 0: Machine Learning 13/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

What is the Learning Problem?

Learning: Improving with experience at some task

Improve over task T

with respect to performance measure P

based on experience E

Example: Learn to play checkers

T : Play checkers

P : % of games won in world tournament

E: opportunity to play against self

Tuo Zhao — Lecture 0: Machine Learning 14/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Predict Emergent C-sections

Data:

2

Learning to Predict Emergency C-Sections

9714 patient records, each with 215 features

[Sims et al., 2000]

Learning to detect objects in images

Example training images for each orientation

(Prof. H. Schneiderman)

One of 18 learned rules:

2

Learning to Predict Emergency C-Sections

9714 patient records, each with 215 features

[Sims et al., 2000]

Learning to detect objects in images

Example training images for each orientation

(Prof. H. Schneiderman)

Tuo Zhao — Lecture 0: Machine Learning 15/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Detect Objects in Images

Tuo Zhao — Lecture 0: Machine Learning 16/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Classify Documents

Tuo Zhao — Lecture 0: Machine Learning 17/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Drive Autonomously

Tuo Zhao — Lecture 0: Machine Learning 18/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Recognize Speech

Tuo Zhao — Lecture 0: Machine Learning 19/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Translate Languages

Tuo Zhao — Lecture 0: Machine Learning 20/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Learn to Play Computer Games

Tuo Zhao — Lecture 0: Machine Learning 21/22

ISYE6740/CSE6740/CS7641: Computational Data Analysis/Machine Learning

Next 3 Lectures

Linear Algebra Review (2 Lectures)

Probability Review (1 Lecture)

Tuo Zhao — Lecture 0: Machine Learning 22/22