supervised learning, categorization and signal-detection ... · categorization and signal-detection...

Supervisedlearning,

categorizationand signal-detectiontheory

Pantelis P.Analytis

Introduction

The linearworld

Instance-basedlearning

Decision trees

Signaldetectiontheory

Supervised learning, categorization andsignal-detection theory

Pantelis P. Analytis

February 28, 2018

1 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


1 Introduction

2 The linear world

3 Instance-based learning

4 Decision trees

5 Signal detection theory

2 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


What is supervised learning?

Classification—categorization problems

Regression—estimation problems

3 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


The lens model: a generic framework

developed by Egon Brunswik in the 30s

Has been adapted and interpreted in different ways bypsychologists.

4 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


The linear lens

Hammond (1955), Todd (1954)

Linear model: y = x0 + x1 · b1 + x2 · b2.... + xn · bn

5 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Actuarial vs. clinical judgement

First comparison of human and machine judgement byPaul Meehl in 1954.

Comparison on the same prediction problem, but possiblyusing different past training data. Use of cross-validationto evaluate the models.

Linear models or even simpler algorithms consistentlyoutperform humans.

6 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Paramorphic representation of judgement

Hoffman (1960) used the linear model to develop the firstmethod of preference learning, precursor to conjointanalysis.

7 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Paramorphic representation of judgement

Hoffman (1960) used the linear model to develop the firstmethod of preference learning, precursor to conjointanalysis.

8 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Logistic regression: simple but robust

b0 shifts the position of the curve and b1 (or b1 + ...bn)defines its slope.

Logistic regression can deal with any number of variables

Computationally very cheap. It is used heavily in theindustry although there are many better algorithms.

9 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Instance-based learning—exemplars and prototypes

First appearance in Statistics: Fix and Hodges (1951) andCover and Hart (1967).First appearance of prototypes in Psychology in 1954 in apaper by Fred Attneave. 10 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


K-nearest neighbors

Define a similarity vector:√∑k

i=1 (xi − yi )2

Pros and cons: intuitive and easy to implement butcomputationally very expensive.

Variations of the basic approach: assign weightsdepending on distance, discard some instances.

11 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


K-nearest neighbors

Define a similarity vector:√∑k

i=1 (xi − yi )2

Pros and cons: intuitive and easy to implement butcomputationally very expensive.

Variations of the basic approach: assign weightsdepending on distance, discard some instances.

12 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Shepard’s Law

Humans and other animals generalize on the basis offeature distance from previously seen items.

Shepard suggested that if there is any law in psychology,that should be it (Science, 1987).

13 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Discussion: do we have access to our ownalgorithms?

Most of the decisions we make are intuitive. What doesexactly intuition mean?

Hogarth (2001) has dedicated a book on the topic.

14 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Medical decision trees

Super (1984), Super triage and rapid treatment (STARTtree).

15 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Bailing out decisions

A model of how British judges decide whether to make apunitive bail (Dahmi, 2003).

16 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


First decision trees

Morgan and Sonquist, 1963, formulated the firstregression tree in Statistics.

Hunt et al. 1966, concept learning system.

17 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


CART, ID3 and C.4.5 algorithm

Friedman (1977), Breiman et al. (1984).

Quinlan (1979,1983,1986,1993).

18 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Shannon’s entropy and information gain

Entropy =∑n

i=1−pi · log2 pi

G (S ,A) = Entropy(S) −∑

v∈Values(A)|Sv ||S | · Entropy(Sv )

19 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Pros and cons of decision trees

Pros: Comprehensible and easy to explain, inexpensiveonce trained, work with continuous and categoricalvariables, can capture non-linearities.

Cons: Weaker in estimation tasks, prone to overfittingespecially with small samples (pruning can somewhatcounteract that), can be costly to train.

20 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Random forests to the rescue

Trained on different samples of data and then aggregatedwith majoritarian voting.

Random forests curb the variance of the algorithm,reducing the overall error.

21 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Signal detection theory

Origins in Neyman-Pearson hypothesis testing, firstapplications in the use of radars.

Introduced in psychology to study perception andsensation (1954).

22 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees

Signaldetectiontheory Criterion: the internal response level beyond which a

decision-maker responds yes (radar operator, doctor orsearch engine)

d’ = separation/spread

23 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Broadly used paradigm in psychophysics.

Also widely employed to define measure of performance ininformation retrieval and recommender systems.

24 / 25

Supervisedlearning,


Pantelis P.Analytis

Introduction

The linearworld


Decision trees


Signal detection theory

Luan et al. (2011) draw connections between the fast andfrugal tree framework and SDT.

25 / 25

supervised learning, categorization and signal-detection ... · categorization and signal-detection...

Documents