cse 291 - pattern recognition - introduction · cse 291 - pattern recognition - introduction henrik...

34
CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego http://www.hichristensen.net H. I. Christensen (UCSD) CSE 291- PR 1 / 34

Upload: dangdieu

Post on 08-Jul-2018

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

CSE 291 - Pattern Recognition - Introduction

Henrik I. Christensen

Computer Science and EngineeringUniversity of California, San Diegohttp://www.hichristensen.net

H. I. Christensen (UCSD) CSE 291- PR 1 / 34

Page 2: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 2 / 34

Page 3: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Introduction

Welcome to CSE 291

Pattern Recognition

Today:

Outline of the course - Objective / MotivationSchedule of lecturesStyle of the courseExercises / ProjectsMaterial to be used in the course

H. I. Christensen (UCSD) CSE 291- PR 3 / 34

Page 4: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Information

Class website:http://www.hichristensen.net/cse291Schedule, Material, Slide copies, General Information

TritonEd - Usual stuff, announcements, ...

Slides - PDF copy will be posted after class with summary

Piazza - You will receive an invitation for the class forumUse it for general questions / discussions

H. I. Christensen (UCSD) CSE 291- PR 4 / 34

Page 5: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Staffing

Henrik I ChristensenLecturer

Ruffin White, TA Priyam Parashar, TA

H. I. Christensen (UCSD) CSE 291- PR 5 / 34

Page 6: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 6 / 34

Page 7: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Objective

Get a solid knowledge of key methods in pattern recognition

Discuss state of the art methods / techniques in pattern recognition

Explore a few representative data sets that illustrate use of patternrecognition

Explore increasingly complex methods over the quarter

This is not a general machine learning course

H. I. Christensen (UCSD) CSE 291- PR 7 / 34

Page 8: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Motivation

PR is used everywhere in daily lives

H. I. Christensen (UCSD) CSE 291- PR 8 / 34

Page 9: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Speech Recognition

H. I. Christensen (UCSD) CSE 291- PR 9 / 34

Page 10: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Financial trading

H. I. Christensen (UCSD) CSE 291- PR 10 / 34

Page 11: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Medical Diagnostics

H. I. Christensen (UCSD) CSE 291- PR 11 / 34

Page 12: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Face Recognition

H. I. Christensen (UCSD) CSE 291- PR 12 / 34

Page 13: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Scene Labeling

H. I. Christensen (UCSD) CSE 291- PR 13 / 34

Page 14: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Netflix Movie Recommendation

H. I. Christensen (UCSD) CSE 291- PR 14 / 34

Page 15: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Generic Problem Structure

Feature Extraction Classification Decision

H. I. Christensen (UCSD) CSE 291- PR 15 / 34

Page 16: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 16 / 34

Page 17: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Structure

A mixture of foundational lectures and

Group discussions of influential/current papers

We will divide class into 3 groups for smaller discussionsEvery student is expected to present 1 paper during term as part of the groupdiscussions

Large group lectures are a challenge for in-depth discussions

H. I. Christensen (UCSD) CSE 291- PR 17 / 34

Page 18: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Lecture topics

1 Bayes Decision Theory

2 Linear Methods for Classification

3 Sub-space Methods

4 Ensemble Methods

5 Hidden Markov Models

6 Prototype/memory based methods

7 Kernels and other tricks

8 Tree based techniques

9 Large Margin Classifiers

10 Deep Learning

H. I. Christensen (UCSD) CSE 291- PR 18 / 34

Page 19: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Discussion Sessions

Discuss two papers per class:

Each paper:

Student presentation of paper ≈ 10 minutes introGroup: What are the main lessons/key insight from the paperGroup: How could it be improved / what would you do differently?TA/Lecturer: guide discussion / presentation

H. I. Christensen (UCSD) CSE 291- PR 19 / 34

Page 20: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 20 / 34

Page 21: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Homework

Leverage of a set of datasets - varying in complexity, ...

A homework assignment roughly every 3 weeks

3 assignments in total

First two will use the common datasets (Gaussian / Ensemble / Temporal)

Final homework - option to use your own dataset - large margin / deeplearning

H. I. Christensen (UCSD) CSE 291- PR 21 / 34

Page 22: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Credit / Grading

40% Homeworks 1-2

30% Homework 3

25% Class Presentation / Discussions

5% Class participation

There is no final exam!

H. I. Christensen (UCSD) CSE 291- PR 22 / 34

Page 23: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Submission of material

Please submit home work on time.

Late submissions will be 75% for 1 day late, 50% for 2 days late and then25% after that

You can ask for permission with a good motivation, but have to do it wellahead of time (not an hour before!)

Do not expect that we are online the last hour before a deadline. Unfair tothe TAs and others.

H. I. Christensen (UCSD) CSE 291- PR 23 / 34

Page 24: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 24 / 34

Page 25: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Book 1: Elements of Statistical Learning

Main textbookElements of Statistical LearningT. Hastie, R. Tibshirani & J. FreiedmanSpringer Verlag, 2nd Edition, 2009http://www-stat.stanford.edu/

~tibs/ElemStatLearn

H. I. Christensen (UCSD) CSE 291- PR 25 / 34

Page 26: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Book 2: Duda, Hart and Stork

Pattern ClassificationR. O. Duda, P. E. Hart and D. G. StorkWiley Interscience, 2nd, 2001, ISBN0-471-05669-3

H. I. Christensen (UCSD) CSE 291- PR 26 / 34

Page 27: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Software

You can use Matlab or Python - we will try to support both

Some demonstrations using Matlab / K. Murphy Toolkit

https://github.com/probml/pmtk3

Some examples using Scikit-Learn Toolkit

http://scikit-learn.org

Still try to finalize 2-3 datasets for homework

H. I. Christensen (UCSD) CSE 291- PR 27 / 34

Page 28: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 28 / 34

Page 29: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Model based recognition

Garmin has 2.5 million objects inthe 3D object warehouse

Can we use these for recognition ofobjects?

Can we provide context for objectrecognition?

H. I. Christensen (UCSD) CSE 291- PR 29 / 34

Page 30: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Gesture based recognition

Tracking of hands for person forrobot interaction

Color classification of hands andhead of user

Tracking of objects using Kalmanfilter

HMM based recognition of gestures

H. I. Christensen (UCSD) CSE 291- PR 30 / 34

Page 31: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Recognition of daily activities

Images of standard objects torecognize daily activities

Example application for assistanceto people with memory challenges

Using Deep Learning forRecognition of situations

H. I. Christensen (UCSD) CSE 291- PR 31 / 34

Page 32: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Outline

1 Introduction

2 Objective / Motivation

3 Schedule / Structure

4 Homework / Exercises

5 Material ...

6 Background examples

7 Questions

H. I. Christensen (UCSD) CSE 291- PR 32 / 34

Page 33: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Next Lecture

Thursday - Bayes Decision Theory

DHS: Chapter 2 (2.1-2.6)

We will provide the initial list of papers for class discussions

Discuss the datasets for home work

H. I. Christensen (UCSD) CSE 291- PR 33 / 34

Page 34: CSE 291 - Pattern Recognition - Introduction · CSE 291 - Pattern Recognition - Introduction Henrik I. Christensen Computer Science and Engineering University of California, San Diego

Questions?

Questions?

H. I. Christensen (UCSD) CSE 291- PR 34 / 34