action rules discovery without pre-existing classification rules b y zbigniew w. ras 1,2 and...

Action Rules DiscoveryAction Rules Discoverywithout pre-existing without pre-existing classification rulesclassification rules

BByy

Zbigniew W. RasZbigniew W. Ras1,21,2 and Agnieszka Dardzinska and Agnieszka Dardzinska1,31,3

1)1) University of North Carolina, Charlotte, NC, USAUniversity of North Carolina, Charlotte, NC, USA2)2) Polish Academy of Science, Warsaw, PolandPolish Academy of Science, Warsaw, Poland3)3) Bialystok Technical University, Bialystok, PolandBialystok Technical University, Bialystok, Poland

1. Introduction : 1. Introduction : Action RuleAction Rule

An action rule is a rule extracted from a datatable that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute

A B D

a1 b2 d1

a2 b2

a2 b2 d2

Information System


An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute

A B D

a1 b2 d1

a2 b2

a2 b2 d2

Information System

Assumption:

- attributes are partitioned into stable and flexible


An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute

A B D

a1 b2 d1

a2 b2

a2 b2 d2

Information System

Assumption:

- attributes are partitioned into stable and flexible

Values of attributes can be changed


Action rule is defined as a term

A B D

a1 b2 d1

a2 b2

a2 b2 d2

Information System

conjunction of fixed condition features shared by both groups

proposed changes in values of flexible features

desired effect of the action

[(ω) ∧ (α → β)] →(ϕ→ψ)

2. Background and Objectives

XX a a bb cc dd

xx11 aa11 bb11 cc11 dd11






xx77 aa11 bb22 cc2 2 dd11


Information System Information System

S = (X,A,V)S = (X,A,V)XX = { = {xx11, , xx22, , xx33, , xx44, , xx55, , xx66, , xx77, , xx88}}AA = {a, = {a, b, c, db, c, d}}

X is a set of objectsX is a set of objectsA is a set of attributes (partial A is a set of attributes (partial functions: X functions: X V) V) VVa a is a set of values of attribute a,is a set of values of attribute a,where a where a A, and V = A, and V = {V{Vaa : a : a A}, A},

Information System SInformation System S

Decision Attribute : {d}Stable Attributes : Ast= {a}Flexible Attributes : Afl = {b,c}

Constraints: Minimum Support 1

Minimum Confidence 2

3. ARED Algorithm (Stable and Flexible Att)

XX a a bb cc dd



xx33 aa22 cc22 dd11



xx66 aa11 bb11 dd22



Decision System

2. Background and Objectives2. Background and Objectives

Action rule generation in earlier work used existing Action rule generation in earlier work used existing classification rules (generated, for instance, by rule classification rules (generated, for instance, by rule discovery algorithms such as LERS or ERID)discovery algorithms such as LERS or ERID)

Proposed AlgorithmProposed Algorithm

1. Extract action rules directly from an information 1. Extract action rules directly from an information system without using pre-existing classification rules.system without using pre-existing classification rules.

2. Extract action rules that have minimal attribute 2. Extract action rules that have minimal attribute involvement.involvement.

Set of rules R with supporting objects

Objects a b c d

x1, x2, x3, x4 0 L

x1, x3 0 L

x2, x4 2 L

x2, x4 1 L

x5, x6 3 L

x7, x8 2 1 H

x7, x8 1 2 H

Objects a b c

x1, x2, x3, x4 0

x1, x3 0

x2, x4 2

x2, x4 1

x5, x6 3

Objects b c

x1, x3 0

x2, x4 2

x2, x4 1

x5, x6 3

Objects b

x2, x4 2

x5, x6 3

c = ?c = 0

Objects b c

x1, x2, x3, x4

Objects b

x1, x3

a = 0

Objects b

x2, x4

a = ?

Objects a b c

x7, x8 2 1

x7, x8 1 2

Objects b c

x7, x8 1

a = 2

Objects b c

x7, x8 1 2

a = ?

Stable Attributes: {a, c}

Flexible Attribute: b

Decision Attribute: d

T1 T2

T3 T4

(T3, T1) : (a = 2) (b, 21) ( d, L H)

(a = 2) (b, 31) ( d, L H)

Objects b

x7, x8 1

c = ? c = 2

Objects b

x7, x8 1

c = ?

Objects b

x1, x2, x3, x4

T5

T6

c = 1

Action Rules Discovery System DEAR

3. Action Rules

Atomic Action Term: (a, a1 → a2 )

attribute a1, a2 ∈ Va

If a1 = a2 then a – stable on a1

3. Action Rules



If a1 = a2 then a – stable on a1Set of Action Terms:

1. If t – atomic action term, then t – an action term2. If t1, t2 – action terms, then t1*t2 – an action term3. If t – action term containing sub-terms (a, a1 → a2 ), (b, b1 →

b2), then a ≠ bDomain of Action Term t: Dom(t) – set of all attribute names listed in t

3. Action Rules



If a1 = a2 then a – stable on a1Action Rule: r = [t1 → t2]

Dom(t2)={d} and Dom(t1) in A

action term

atomic action term

3. Example

XX a a bb cc dd









(a, a2 → a2) (b, b1 → b3)(c, c2 → c2)(d, d1 → d2)

Decision System SDecision System S

atomic action terms

r=[[(a, a2→a2)*(b, b1→b3)] → (d, d1→d2)

action rule

Dom (r) ={a, b, d}

4. Interpretation NS of action terms

1. If (a, a1 → a2) – atomic action term then:NS((a, a1 → a2))=[{x X:a(x)=a1∈ }, {x∈X:a(x)=a2}]

2. If t1=(a, a1 → a2) * t and NS(t)=[Y1, Y2], then:

NS(t1)=[ Y1 ∩ {x∈X:a(x)=a1}, Y2 ∩ {x∈X:a(x)=a2} ]

Information System S=(X, A, V)Information System S=(X, A, V)

Assume that [Y1, Y2]∩[Z1, Z2] = [Y1∩Z1, Y2∩Z2], and

NS(t1)=[Y1, Y2],NS(t2)=[Z1, Z2], then:NS(t1*t2)=NS(t1)∩NS(t2)

4. Interpretation NS of action terms

Information System S=(X, A, V)Information System S=(X, A, V)

NS(t1)=[Y1, Y2]

NS(t2)=[Z1, Z2]

Action Rule: r = [t1 → t2]

Support:

Confidence:

)(

)(

)(

)()(

2

22

1

11

Ycard

ZYcard

Ycard

ZYcardrconf

)()sup( 11 ZYcardr

XX a a bb cc dd










r = [[(a, a2→a2)*(b, b1→b2)] → (d, d1→d2)

NS((a, a2→a2))=[{x2, x3, x4, x5}, {x2, x3, x4, x5}] NS((b, b1→b2))=[{x1, x2, x4, x6}, {x3, x7, x8}] NS((d, d1→d2))=[{x1, x2, x3, x4, x5, x7}, {x6}]

XX a a bb cc dd










r = [[(a, a2→a2)*(b, b1→b2)] → (d, d1→d2)

NS((a, a2→a2))=[{x2, x3, x4, x5}, {x2, x3, x4, x5}] NS((b, b1→b2))=[{x1, x2, x4, x6}, {x3, x7, x8}] NS((d, d1→d2))=[{x1, x2, x3, x4, x5, x7}, {x6}]

NS((a, a2→a2)*(b, b1→b2))=[{x2, x4}, {x3}]

sup (r) = 2, conf (r) = 0

5. Algorithm ARED

-Similar to ERID and LERS-Only positive marks yield action rules-Action terms of length k are built from unmarked action terms of length k – 1 and unmarked atomic action terms of length 1

S=(X, A ∪ {d}, V) – decision systemλ1, λ2 – minimum support and confidenceCS(a)={NS(ta): ta – an atomic action term built from elements in Va}

a ∈ A

5. ARED Algorithm

For decision attribute in S:

NS(d,d1 d2)=[{x1,x2, x3, x4, x5, x7}, {x6}]

XX a a bb cc dd









stable attribute

flexible attributes

Object reclassification from class d1 to d2 λ1=2, λ2=1/4

Notation: t1=(b,b1b1), t2=(b,b2b2), t3=(b,b3b3), t4=(a,a1a2), t5=(a,a1a1), t6=(a,a2a2), t7=(a,a2a1), t8=(c,c1c2), t9=(c,c2c1), t10=(c,c1c1),t11=(c,c2c2),

t12 = (d,d1 d2),

5. ARED Algorithm


NS(d,d1 d2)=[{x1,x2, x3, x4, x5, x7}, {x6}]


For classification attribute in S:

NS(t1)=[{x1,x2, x4, x6}, {x1,x2, x4, x6}]

Not marked λ1=3

NS(t2)=[{x3,x7, x8}, {x3,x7, x8}] Mark “-” λ1=1

NS(t3)=[{x5}, {x5}] Mark “-” λ1=1

NS(t4)=[{x1,x6, x7, x8}, {x2,x3, x4, x5}]

Mark “-” λ2=0

NS(t5)=[{x1,x6, x7, x8}, {x1,x6, x7, x8}]

Not marked λ1=4

NS(t6)=[{x2,x3, x4, x5}, {x2,x3, x4, x5}]

Mark “-” λ2=0

Mark “+” λ1=4,λ2=1/4 NS(t7)=[{x2,x3, x4, x5}, {x1,x6, x7, x8}]

5. ARED Algorithm


NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]



NS(t1)=[{x1,x2, x4, x6}, {x1,x2, x4, x6}]

Not marked λ1=3

NS(t2)=[{x3,x7, x8}, {x3,x7, x8}] Marked “-” λ2=0

NS(t3)=[{x5}, {x5}] Marked “-” λ1=1

NS(t4)=[{x1,x6, x7, x8}, {x2,x3, x4, x5}]

Marked “-” λ2=0

NS(t5)=[{x1,x6, x7, x8}, {x1,x6, x7, x8}]

Not marked λ2=2

NS(t6)=[{x2,x3, x4, x5}, {x2,x3, x4, x5}]

Marked “-” λ2=0

Mark “+” λ1=4, λ2=1/4NS(t7)=[{x2,x3, x4, x5}, {x1,x6, x7, x8}] r = [t7 t1]

5. ARED Algorithm


NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]



NS(t8)=[{x1,x4, x8}, {x2, x3, x5, x6, x7}]

Not marked

Marked “-”

NS(t10)=[{x1, x4, x8}, {x1, x4, x8}] Marked “-”

NS(t11)=[{x2, x3, x5, x6, x7}, {x2, x3, x5, x6, x7}]

Not marked

Rule r = [t8→t12], conf = 2/3 *1/5 <λ2, sup=2 ≥ λ1

NS(t9)=[{x2, x3, x5, x6, x7}, {x1, x4, x8}]

5. ARED Algorithm


NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]



Marked “+”

NS(t1*t11)=[{x2, x6}, {x2, x6}] Not marked

NS(t5*t8)=[{x1, x8}, {x6, x7}] Marked “-”

Rule r = [t1*t8→t12], conf = 1/2 ≥ λ2, sup=2 ≥ λ1

Now action terms of length 2 from unmarked action terms of length 1


NS(t1*t8)=[{x1, x4}, {x2, x6}]

Rule r = [t5*t8→t12], conf = 1/2 ≥ λ2, sup= 1 < λ1


NS(t8*t11)=[Ø, {x2, x3, x5, x6, x7}] Marked “-”

5. ARED Algorithm


NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]



Now action terms of length 3 from unmarked action terms of length 2

(t1*t5*t8) –extension of (t5*t8)

Marked “-”

Algorithm STOPS!

5. ARED Algorithm


NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]



Action rules:

[[(b,b1→b1)*(c,c1→c2)] → (d, d1→d2)]

[[(a,a2→a1] → (d, d1→d2)]

XX a a bb cc dd









6. Application

Database HEPAR (Medical Center of Postgraduate Education, Warsaw, Poland) prepared by Dr. med. Hanna Wasyluk

758 758 patientspatients described by 106 attributes (including 31 laboratory tests described by 106 attributes (including 31 laboratory tests with values discretized to: “below normal”, “normal”, “above normal”). with values discretized to: “below normal”, “normal”, “above normal”). 14 attributes are stable. Two tests are invasive tests: 14 attributes are stable. Two tests are invasive tests: HBsAg [in tissue], HBcAg [in tissue]. There are 7 decision values:

I. I. Acute hepatitisAcute hepatitisIIa. IIa. Subacute hepatitis (types BC)IIb. IIb. Subacute hepatitis (alcohol-abuse)IIIa.IIIa. Chronic hepatitis (curable)IIIb.IIIb. Chronic hepatitis (non-curable)IV.IV. Cirrhosis hepatitisV.V. Liver cancer

We can ask for re-classifications: IIb We can ask for re-classifications: IIb I, IIIa I, IIIa I I

6. Application

Database HEPAR (heavily incomplete)

Handling data incompletenessHandling data incompleteness

Attributes with more than Attributes with more than 90% 90% of null values have been of null values have been removedremoved

Subjective attributes have been removedSubjective attributes have been removed [example: [example: History of alcohol abuse]

Removal of absolute medical testsRemoval of absolute medical tests

RSES RSES null value imputation module was used to make the null value imputation module was used to make the resulting dataset complete.resulting dataset complete.

6. Application

ChosenChosen redu reducct t has no invasive tests has no invasive tests, 11% , 11% incomplete dataincomplete data{m, n, q, u, y, aa, ah, ai, am, an, aw, bb, bg, by, cj, cm}

m Bleedingn Subjaundice symptomsq Eructationu Obstructiony Weight lossaa Smoking ah History of viral hepatitis (stable)ai Surgeries in the past (stable)am History of hospitalization (stable)an Jaundice in pregnancy aw Erythematous dermatitisbb Cysts bg Sharp liver edge (stable)bm Blood cell plaqueby Alkaline phosphatasecj Prothrombin index cm Total cholesteroldd Decision attribute

6. ApplicationAction Rules discovery in Database HEPAR

No history of viral hepatitis butbut withwith history of shistory of surgery and hospitalization, , ssharp liver edge normal, , no sno subjaundice symptoms, , ttotal cholesterol normal, , eerythematous dermatitis normal, wweight normal, no cysts , patient does not smoke

[(ah=1)[(ah=1) (ai=2) (ai=2) (am=2) (am=2)(bg=1)](bg=1)](cm=1)(cm=1) (aw=1) (aw=1)(u, (u, 1) 1)(bb=1)(bb=1) (aa=1) (aa=1) (q, (q, 1) 1) (m, (m, 1) 1) (n=1) (n=1) (bm, (bm, down) down) (y=1) (y=1) (by, norm (by, norm up) up) (dd, IIIa (dd, IIIa I ) I )

[[(ah=1)(ah=1) (ai=2)(ai=2) (am=2) (am=2) (bg=1)] (bg=1)](cm=1)(cm=1)(aw=1)(aw=1)(u, (u, 1) 1) (bb=1) (bb=1) (aa=1) (aa=1) (q, (q, 1) 1)(m, (m, 1) 1)(n=1)(n=1)(bm, (bm, down) down)(y=1)(y=1) (by, norm (by, norm down) down)

(dd, IIIa (dd, IIIa I ) I ) Two quite similar action rules have been constructedTwo quite similar action rules have been constructed::

By getting rid of obstruction, eructation, bleeding, by decreasing the bBy getting rid of obstruction, eructation, bleeding, by decreasing the blood cell plaque and by changing the level of a and by changing the level of alkaline phosphatase we should be able we should be able reclassify the patient from IIIa group to Ireclassify the patient from IIIa group to I. . Medical doctor should decide for a given patient if Medical doctor should decide for a given patient if the alkaline phosphatase level level needs to be decreased or increasedneeds to be decreased or increased. .

Attribute values of Attribute values of ttotal cholesterol, wweight, and smoking have to stay unchangedhave to stay unchanged. .

• Presented algorithm discovers action rules directly from a decision table.

• The proposed algorithm generates the shortest action rules without using pre-existing classification rules

• Future work includes- Action rule generation from incomplete information system

- Action rule generation when attributes are hierarchical

- Considering different flexibilities of attributes (e.g. the social condition is most likely less flexible than

the health condition )

7. Conclusion and Future Work

Thank You

Questions?

action rules discovery without pre-existing classification rules b y zbigniew w. ras 1,2 and...

Documents