chapter 5 decision tree induction using frequency tables for attribute selection
DESCRIPTION
Decision Tree Induction Using Frequency Tables for Attribute SelectionTRANSCRIPT
![Page 1: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/1.jpg)
Nguyễn Dương Trung Dũng
1
![Page 2: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/2.jpg)
Content 1. Calculating Entropy in Practice
2. Gini Index of Diversity
3. Inductive Bias
4. Using Gain Ratio for Attribute Selection
2
![Page 3: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/3.jpg)
Calculating Entropy in Practice Age specRx astig tears Class
1 1 1 1 3
1 1 1 2 2
1 1 2 1 3
1 1 2 2 1
1 2 1 1 3
1 2 1 2 2
1 2 2 1 3
1 2 2 2 1
Training Set 1 (age=1) for lens24
3
![Page 4: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/4.jpg)
Calculating Entropy in Practice age=1 age=2 age=3
Class 1 2 1 1
Class 2 2 2 1
Class 3 4 5 6
Column Sum 8 8 8
Frequency Table for Attribute age for lens24
The cells of this table show the number of occurences of each combination of class and attribute value in the training set.
4
![Page 5: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/5.jpg)
Calculating Entropy in Practice
5
![Page 6: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/6.jpg)
Gini Index of Diversity
6
![Page 7: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/7.jpg)
Gini Index of DiversityAge specRx astig tears Class
1 1 1 1 3
1 1 1 2 2
1 1 2 1 3
1 1 2 2 1
1 2 1 1 3
1 2 1 2 2
1 2 2 1 3
1 2 2 2 1
Training Set 1 (age=1) for lens24
7
![Page 8: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/8.jpg)
Gini Index of Diversity
8
![Page 9: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/9.jpg)
Gini Index of DiversityWe can now calculate the new value of the Gini Index as follows
(a) For each non-empty column, form the sum of the squares of the values in the body of the table and divide by the column sum.
(b) Add the values obtained for all the columns and divide by N (the number of instances)
(c) Subtract the total from 1
9
![Page 10: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/10.jpg)
Gini Index of Diversityage=1 age=2 age=3
Class 1 2 1 1
Class 2 2 2 1
Class 3 4 5 6
Column Sum 8 8 8
10
![Page 11: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/11.jpg)
Gini Index of Diversity
11
![Page 12: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/12.jpg)
Inductive Bias
12
![Page 13: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/13.jpg)
Inductive Bias Inductive bias:
- A preference for one choice rather than another
- Determined by external factors such as our preferences, simplicity, familiarity
- Any formula we use for it introduces an inductive bias
13
![Page 14: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/14.jpg)
Using Gain Ratio for Attribute Selection
14
![Page 15: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/15.jpg)
Using Gain Ratio for Attribute Selection
age=1 age=2 age=3
Class 1 2 1 1
Class 2 2 2 1
Class 3 4 5 6
Column Sum 8 8 8
15
![Page 16: Chapter 5 decision tree induction using frequency tables for attribute selection](https://reader033.vdocuments.us/reader033/viewer/2022052602/559e9f7a1a28abde048b47c4/html5/thumbnails/16.jpg)
The end
16