national centre for agricultural economics and policy research (ncap), new delhi rajni jain...
TRANSCRIPT
National Centre for Agricultural Economics and Policy Research (NCAP), New Delhi
National Centre for Agricultural Economics and Policy Research (NCAP), New Delhi
Rajni [email protected]
Expert System: Major Constraint
• Expert systems are automated systems that provide knowledge like experts and are useful when it is difficult to get expert advice
• Development of expert system is challenging because it is to hard to develop the complete rule base which can work like an expert:– Non availability of experts– Scarcity of time– Cost of their time
DM: An aid for development of Expert System
• DM: Extraction of potentially useful, meaningful and novel patterns from the data using suitable algorithms
• It is a great idea to use data mining tools and techniques to extract rules.
• reduce the time required for interactions with the domain experts
• Rules should never be used without validation from experts
Introduction
Importance of Rules
Directly from Decision Tables
DM
– DT– RS– Fuzzy– ANN
Decision TREE
•Used to represent a sequence of decisions to arrive at a particular result
Y N
ColourWhite
Red
Decision-Tree (ID3)A decision-tree is a set of nodes and leaves
where each node tests the value of an attribute and branches on all possible values.ID3 is the first most popular decision-tree building
algorithm.A statistical property called information gain is
used to decide which attribute is best for the node under consideration.Greedy Algorithm, all attributes till the examples
reach leaf node are retained for computation. Parent node attribute is eliminated.
CLS (Concept Learning System)
Step 1: If all instances in C are positive, then create YES node and
halt.– If all instances in C are negative, create a NO node and halt.– Otherwise select a feature, F with values v1, ..., vn and create a decision node.
Step 2: Partition the training instances in C into subsets C1, C2, ..., Cn
according to the values of V.
Step 3: apply the algorithm recursively to each of the sets Ci.
CLS contd..
Note, the trainer (the expert) decides which feature to
select.
ID3 Algorithm
ID3 improves on CLS by adding a feature selection heuristic.
ID3 searches through the attributes of the training instances and extracts the attribute that best separates
the given examples. If the attribute perfectly classifies the training sets then ID3 stops; otherwise it recursively
operates on the n (where n = number of possible values of an attribute) partitioned subsets to get their "best"
attribute.
The algorithm uses a greedy search, that is, it picks the best attribute and never looks back to reconsider
earlier choices.
Data Requirements for ID3
Attribute value Description: The same attributes must describe
each example and have a fixed number of values
Predefined classes: An example’s attributes must already be
defined as they are not learned by ID3
Discrete Classes
Sufficient Examples
Attribute Selection in ID3
How does ID3 decides which attribute is the best?– Information Gain, a statistical property is used
Information gain measures how well a given attribute separates training
examples into targeted classes
The attribute with the highest information gain is selected
Entropy measures the amount of the information
Entropy
Given a collection S of c outcomes
Entropy(S) = -p(I) log2 p(I) where p(I) is the proportion of S belonging to class I. is over c. Log2 is log base 2.Note that S is not an attribute but the entire sample set.
Example 1: Entropy
If S is a collection of 14 examples with 9 YES and 5 NO examples then
Entropy(S) = - (9/14) Log2 (9/14) - (5/14) Log2 (5/14) = 0.940
Notice entropy is 0 if all members of S belong to the same class (the data is perfectly classified). The range of entropy is 0 ("perfectly classified") to 1 ("totally random").
Information Gain
Gain(S, A) is information gain of example set S on attribute A is defined as
Gain(S, A) = Entropy(S) - ((|Sv| / |S|) * Entropy(Sv))
Where:is each value v of all possible values of attribute A
Sv = subset of S for which attribute A has value v
|Sv| = number of elements in Sv
|S| = number of elements in S
Example 2: Information Gain
Suppose S is a set of 14 examples in which one of the attributes is wind speed. The values of Wind can be Weak or Strong. The classification of these 14 examples are 9 YES and 5 NO. For attribute Wind, suppose there are 8 occurrences of Wind = Weak and 6 occurrences of Wind = Strong. For Wind = Weak, 6 of the examples are YES and 2 are NO. For Wind = Strong, 3 are YES and 3 are NO. Therefore
Gain(S,Wind)=Entropy(S)-(8/14)*Entropy(Sweak)-(6/14)*Entropy(Sstrong) = 0.940 - (8/14)*0.811 - (6/14)*1.00= 0.048
Entropy(Sweak) = - (6/8)*log2(6/8) - (2/8)*log2(2/8) = 0.811
Entropy(Sstrong) = - (3/6)*log2(3/6) - (3/6)*log2(3/6) = 1.00
For each attribute, the gain is calculated and the highest gain is used in the decision node.
Example 3: ID3
Suppose we want ID3 to decide whether the weather is amenable to playing baseball. Over
the course of 2 weeks, data is collected to help ID3 build a decision tree (see table 1).
The target classification is "should we play baseball?" which can be yes or no.
The weather attributes are outlook, temperature, humidity, and wind speed. They can have
the following values:
outlook = { sunny, overcast, rain }
temperature = {hot, mild, cool }
humidity = { high, normal }
wind = {weak, strong }
Day Outlook Temperature
Humidity Wind Play ball
D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No
Real Example:Summary of dataset
NSSO carried out a nationwide survey on cultivation practices as a part of its 54th round from Jan-June 1998The enquiry was carried out in the rural areas of
India through house hold survey to ascertain the problems and utilization of facilities and the level of adoption in farms of different sizes in the countryFocus on the spread of improved agricultural
technologyData is extracted for the Haryana State
Dataset Summary
39 independent variables from Haryana1 dependent variable (ifpesticide) value 1,2
– 1:adoption of PIF– 2:non-adoption of PIF
36-nominal, 4 real-valued attributes1832 cases
– 48% adopting PIF – 52% not adopting
Why Haryana?
• Reasonable dataset size
(1832 cases)
• Independent variable (if pesticide) is some what uniformly distributed in Haryana
Lakshdweep 0 Arunachal 35
Mizoram 1 Bihar 40
Sikkim 2 Maharastra 43
Dadra 8 A&N 43
Nagaland 11 Haryana 48
Himachal 14 Goa 52
Meghalaya 16 Karnatka 55
Kerala 20 Tripura 58
UP 23 Gujrat 65
Rajasthan 24 Punjab 71
MP 27 DamanDiu 74
J&K 29 Andhra 79
Manipur 31 TN 80
Assam 32 WB 82
Orissa 32 Pondicherry 88
Delhi 33 Chnadigarh 100
Distribution of Class Variable
Variables in Haryana-Farmers Dataset (NSSO, 54th round)
# Attribute Name Value set # Attribute Name Value set
1 SnoCropGroup {1,2,3,4,5} 21 jointforest {1,2}
2 CropCode {1,2,3,4,5,6,7,8,9,11,99} 22 managetanks {1,2}
3 Season {1,2} 23 iftreepatta {1,2}
4 NumAreaSown {1,2,3,4,5,6} 24 iftimberright {1,2}
5 iftractor {1,2,3} 25 howoftenright {1,2,9}
6 ifepump {1,2,3} 26 schhhdcpr {1,2,9}
7 ifoilpump {1,2,3} 27 anymemberpreventcpr {1,2}
8 ifmanure {1,2,3} 28 LandWithRightOfSale {1,2,3,4,5}
9 iffertilizers {1,2,3} 29 TotLandOwnedNum {1,2,3,4,5}
10 ifimproveseed {0,1,2,3} 30 LandPossessNum {1,2,3,4,5}
11 seedtype {1,2,3,4,9} 31 NASNum {1,2,3,4,5}
12 typemanure {1,2,3,9} 32 ifsoiltested {1,2}
13 ifhired {1,2,9} 33 ifsoilRecfollow {1,2,9}
14 ifirrigate {1,2} 34 ifhhdtubewell {1,2}
15 ifhiredirri {1,2,9} 35 iftubewellunused {1,2,9}
16 ifweedicide {1,2} 36 irrigatesource {1,2,9}
17 ifunusdelectricpump {1,2,9} 37 ifdieselpump {1,2}
18 harvested {1,2,3} 38 ifunuseddieselpump {1,2,9}
19 woodpurpose {1,2,3,4,5,9} 39 ifelectricpump {1,2}
20 iflivestock {1,2} 40 ifpesticide {1,2}
Hypothetical Farmer Dataset
ID Crop Seedtype AreaSown Iffertilizer Ifpesticide
X1 wheat 2 small no yes
X2 wheat 3 medium yes no
X3 wheat 1 medium yes no
X4 rice 1 medium no yes
X5 fodder 2 large no yes
X6 rice 3 large no no
X7 rice 2 large no no
X8 wheat 1 small yes no
Decision Tree (DT) using RDT Architecture for Hypothetical Farmer
Data
no
Seedtype
1
Cropricefodderwheat
no yes?
Cropricefodderwheat
yes noyes
2 3
•The Tree is simple to comprehend•The tree can be easily traversed to generate rules
Description of the codes of the attributes in the Approximate Core based RDT model
Attribute name # Code specifications
Ifweedicide 16 yes -1, no-2
CropCode 2 paddy -1, wheat -2, other cereals -3, pulses- 4, oil seeds -5, mixed crop – 6, sugarcane -7, vegetables -8,
fodder -9, fruits & nuts -10, other cash crops -11, others 99
Iffertilizer 9 entirely -1, partly-2, none-3
Ifpesticide 40 yes-1, no-2
If weedicide
11
1CropCode
1
1 22
2
2 3
2
2
4
2
2
5
2
2
26 7
99
8
9
2 2 2
2 2 2
Iffertilizer
12
2
1
31
Iffertilizer
11
1
2
1
32
11
Approximate Core based RDT model for Farmers Adopting Plant Protection Chemicals
1
Ifweedicide: yes -1, no-2
Ifpesticide:yes-1, no-2
Cropecode: paddy -1, wheat -2, other cereals -3, pulses- 4, oil seeds -5, mixed crop – 6, sugarcane -7, vegetables -8,fodder -9, fruits & nuts -10, other cash crops -11, others 99
Iffertilizer: entirely -1, partly-2, none-3
Decision Rules for Adopters1. If Weedicide= yes then pesticide=yes
2. If weedicide=no and crop=paddy then pesticide=yes
3. If weedicide=no and crop=vegetables and fertilizer=entirely then pesticide=yes
4. If weedicide=no and crop=vegetables and fertilizer=none then pesticide=yes
5. If weedicide=no and crop=cash and fertilizer=entirely then pesticide=yes
6. If weedicide=no and crop=cash and fertilizer=partly then pesticide=yes
Decision Rules for Non-adopters
1.If weedicide=no and crop=(wheat, other than rice cereals, pulses, oil seeds, mixed crop, sugarcane, fodder, other crops) then pesticide=no
2.If weedicide=no and crop = vegetables and fertilizer=partly then pesticide=no
3.If weedicide=no and crop = other cash crops and fertilizer=none then pesticide=no
A case study of forwarning Powdery Mildew of Mango disease
YEAR T811 H811 T812 H812 T813 H813 T814 H814STATUS
1987 28.20 91.25 28.48 88.60 29.50 85.50 30.14 82.86 1
1988 32.05 75.75 31.64 80.60 31.27 77.83 30.66 79.57 0
1989 26.60 86.25 26.28 87.20 26.47 87.33 26.31 89.14 0
1990 27.50 91.25 28.12 91.20 28.17 92.00 28.43 91.00 1
1991 28.43 87.00 28.70 86.20 29.00 83.50 29.57 80.57 0
1992 30.12 69.23 30.45 68.58 30.80 68.31 31.25 67.82 1
1993 30.50 61.75 30.48 61.13 30.37 60.56 30.33 61.76 0
1994 30.45 89.25 30.56 85.80 30.63 83.17 30.71 81.14 1
1995 28.63 61.38 29.10 61.20 29.58 61.17 30.71 61.57 0
1996 31.63 60.33 31.90 60.87 32.67 60.89 33.07 59.76 1
1997 32.13 71.00 32.20 69.40 31.67 69.00 31.50 68.29 0
2000 29.00 78.33 29.23 78.60 29.36 78.83 29.52 79.14 0
Dataset: PWM status for 11 years and corresponding temperature and humidity conditions
Techniques for Rule based learning
NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH
OrThe hybrids likeRDT
Selection of Algorithms
• Accuracy: Fraction of test instances which are predicted accurately
• Interpretability: Ability to understand• Complexity: No of conditions in the model• No of rules• No f variablesSelection of an algorithm involves trade-off with
the parameters
ID TRAIN TEST
1987-94 1987-94 1995, 1996, 1997, 2000
1987-95 1987-95 1996, 1997, 2000
1987-96 1987-96 1997, 2000
1987-97 1987-97 2000
Train and Test Data
The Prediction model for PWM Epidemic as obtained using CJP Algorithm on 1987-97 data as the training
dataset
NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH
<=87 >87
<=30.7 >30.7
Rules
NATIONAL CENTRE FOR AGRICULTURAL ECONOMICS AND POLICY RESEARCH
•If (H811>87) then Status = 1
•If (H811<=87) and (T814<=30.71) then Status = 0
•If (H811<=87) and (T814>30.71) then Status = 1
Conclusions
• Rule induction can be very useful for development of expert systems
• Strong linkage and collaborations are required among expert system developer, domain experts and data mining experts
