Decision Procedures

Manoj Kumar RautTata Institute of Fundamental Research

November 30, 2005

Definition

• A decision procedure is an algorithm for determiningwhether a formula is valid or satisfiable in a logicaltheory

• A theory is any set of sentences,Theories are defined Axiomatically

1

Decision Procedure for TE

• Uninterpreted function symbols, constants,Predicate symbol =Variables, Logical connectives

• Axioms∀x. x = x (Reflexivity)∀xy. x = y → y = x (Symmetry)∀xyz. x = y ∧ y = z → x = z (Transitivity)∀x1 . . . xny1 . . . ynx1 = y1, . . . , xn = yn → f(x1, . . . , xn) =f(y1, . . . , yn) (Congruence)

2

Directed Acyclic Graph

• A DAG is a vertex labeled directed acyclic multigraph

• Nodes represent ground terms and edges are ordered

• If there is an edge from u to v, then u a parent ofv and v a child of u.

• A term f(t1, . . . , tn) is represented by node u if uis labeled as f and children(u) is a sequence ofnodes t1, . . . , tn

3

Representation of E-DAG

• Equalities in DAG is represented as a separate edge.

• Equalities define a relation R on DAG nodes

• f(a, b) = a defines a relationR = (f(a, b), a)

• Example

f(a, b) = a

f

a b

v1

v2 v3

4

Computing Equivalence Closure

• R be any relation. Equivalence closure of R is thesmallest relation RE such that R ⊆ RE and– for all nodes n, (n, n) ∈ RE– (n1, n2) ∈ RE ⇒ (n2, n1) ∈ RE– (n1, n2), (n2, n3) ∈ RE ⇒ (n1, n3) ∈ RE

• We choose arbitrary representatives of equivalenceclasses

– n∗ is the representative of n under the relationRE

– (n1, n2) ∈ RE iff n∗1 = n∗2 (some representative)

5

Computing Congruence Closure

• RE be an equivalence relation on the nodes of aE-DAG

– Congruence closure of an equivalence relationRE is the smallest relation RC ⊇ RE such thatfor all nodes f(n1, . . . , nk) and f(m1, . . . ,mk)in the E-DAG if (ni,mi) ∈ RC implies(f(n1, . . . , nk), f(m1, . . . ,mk)) ∈ RC

6

Example

• f(a, b) = a→ f(f(a, b), b) = a

• Represent all the terms in E-DAGf

f

a b

v1

v2

v3 v4

2

2

1

1

• Observe thatf(f(a, b), b) is congruent to f(a, b)

7

Cont.

f

f

a b

v1

v2

v3 v4

f(f(a, b), b) = af(a, b) = a

f

f

a b

v1

v2

v3 v4

Congruence Closure2

2

1

1

infers

congruent

• Congruence closure is an inference procedure forequality

• Congruence closure always terminates because wedo not add nodes

8

The Procedure

(i) Given F ≡ t1 = t′1 ∧ . . . ∧ tp = t′p ∧

u1 6= u′1 ∧ . . . ∧ uq 6= u′q

(ii) Represent all terms in the E-DAG

(iii) Create R = {(t1, t′1), . . . , (tp, t′p)}

(iv) Close R under equivalences and congruences

(v) Pick representative for each class in RC

(vi) If u∗j = u′j

∗for 1 ≤ j ≤ q implies F is unsatisfiable

(vii) Otherwise F is satisfiable

9

Soundness

TheoremF is satisfiable ⇒ u∗j 6= u

′j

∗for 1 ≤ j ≤ q

Proof Outline

• Let ψ be an interpretation that satisfies F

• Show t∗ = t′∗ ⇒ ψ(t) = ψ(t′)

• Proof by Induction on the number of steps in theconstruction of the congruence closure

10

Cont.

• Base step(t, t

′) ∈ R⇒ ψ(t) = ψ(t′)

• Induction stepAssume ∀(u, u′) ∈ R′ ⇒ ψ(u) = ψ(u′)

Case I: LetR′′

= R′ ∪ {(n1, n3) | (n1, n2) ∈ R′, (n2, n3) ∈ R′}

Show that∀(u, u′) ∈ R′′ ⇒ ψ(u) = ψ(u′)

Case II: LetR′′

= R′ ∪ {(f(t), f(t′) | (t, t′) ∈ R′}

ψ(f(t)) = ψ(f)(ψ(t)) = ψ(f)(ψ(t′)) = ψ(f(t

′))

11

Completeness

TheoremFor 1 ≤ j ≤ q, u∗j 6= u

′j

∗ ⇒ F is satisfiable

• To show F satisfiable,we must show an universe and an interpretation ψsuch that ψ(ti) = ψ(ti

′) and

ψ(uj) 6= ψ(uj ′)

• Universe is the set of representatives in the E-DAG

• ψ(t) = t∗ if t is represented in the E-DAG

ψ(f)(n∗1, . . . , n∗k) =

f(n∗1, . . . , n∗k)∗ if f(n∗1, . . . , n

∗k)

is representedin E-DAG

arbitrary otherwise

12

Cont.

• ψ(ti) = ψ(ti′) because ti∗ = ti′∗

because (ti, ti′) ∈ R

• ψ(uj) 6= ψ(uj ′) because uj∗ 6= uj ′∗

by hypothesis

• Note that ti, ti′, uj, uj ′ are represented byconstruction of E-DAG

13

Decidability

• Validity problem is undecidable(Church, 1936), (Turing, 1936)

• Quantifier free validity problem is decidable(Ackermann, 1954), (Downey et.al. 1980),(Nelson and Oppen, 1980), (Shostak, 1978)

• ComplexityO(nlogn) for a conjuction of literals of length n

14

Theory of List structures, TL

• Function symbols :- Uninterpreted functionscons, car, cdrcons(x, y) is the ordered pair (x, y)car(x, y) = x and cdr(x, y) = y

• Predicates :- atomatom(z) is true iff z is not an ordered pair

• Variables, Logical symbols

• Examplecar(x) = car(y) ∧ cdr(x) = cdr(y) ∧¬atom(x) ∧ ¬atom(y) → f(x) = f(y)

15

Cont.

• Axioms– ∀x∀y car(cons(x, y)) = x– ∀x∀y cdr(cons(x, y)) = y– ∀x ¬atom(x) → cons(car(x), cdr(x)) = x– ∀x∀y ¬atom(cons(x, y))

16

Extending Congruence Closure to a Sat.procedure for TL

• Consider the theory with car, cdr and cons and thefollowing two axioms

• Axiom– ∀x∀y car(cons(x, y)) = x– ∀x∀y cdr(cons(x, y)) = y

17

Example

• Consider the following formula F– x = cons(u, v) ∧ cons(car(x), cdr(x)) 6= x

• Represent the terms in E-DAG

v xu

cons car cdr

cons

18

Cont.

• Add x = cons(u, v)Use axiom 1: car(x) = uUse axiom 2: cdr(x) = v

v xu

cons car cdr

cons

1 2

1 2

19

Cont.

• Use congruence:cons(u, v) = cons(car(x), cdr(x))

v xu

cons car cdr

conscongruent

20

Cont.

• Use equivalence,add cons(car(x), cdr(x)) = x

v xu

cons car cdr

cons

contradiction

• Check disequalities ⇒ contradiction

21

Remarks

• This algorithm seems to work

• The algorithm discovers only necessary equalities– Whenever the algorithm says UNSAT., the

formula is UNSAT.– The algorithm is sound for use as a UNSAT.

procedure.

• The algorithm does not discover all impliedequalities.

22

Example

• F ≡ cons(x, y) = cons(u, v) ∧ x 6= u

x y u v

n1 n2cons cons

• Close it under equivalences and axioms

• Notice that x∗ 6≡ u∗, hence report satisfiability

23

Cont.

• But take any interpretation Ψ that satifies theaxioms

– Ψ(car)(n1) = Ψ(x) (by Axiom 1)– Ψ(car)(n2) = Ψ(u) (by Axiom 2)

x y u v

n1 n2

car

cons cons

24

Cont.

• n1 = n2 ⇒ Ψ(x) = Ψ(u)

• Ψ can not satisfy x∗ 6= u∗,hence unsatisfiable

x y u v

n1 n2

car

cons cons

contradiction

25

Remarks

• The algorithm does not discover this equality andhence incomplete.

• It is because E-DAG did not have enough nodes toallow the closure to discover all the equalities

26

Restoring Completeness

Solution

1. Extend the axioms with∀x∀y∀u∀v cons(x, y) = cons(u, v)→ x = u ∧ y = vor

2. Require that whenever cons(x, y) is representedthen car(cons(x, y)) and cdr(cons(x, y)) are alsorepresented.

27

The Procedure

(i) Given F ≡ ATOM(u1) ∧ . . . ∧ATOM(uq) ∧v1 = w1 ∧ . . . ∧ vr = wr ∧ x1 6= y1 ∧ . . . ∧ xs 6= ys(The terms contain uninterpreted functions andinterpreted functions CAR, CDR, and CONS.)

(ii) Convert ¬ATOM(u) to u = cons(u1, u2)

(iii) Represent all terms in the E-DAG

(iv) Create R = {(vi, wi)} for 1 ≤ i ≤ r

(v) Close R under equivalences and congruences

28

Cont.

(vi) Given a term CONS(x, y), add verticesrepresenting CAR(CONS(x, y)) (= x) andCDR(CONS(x, y)) (= y) in the E-DAG

(vii) Close it under equivalences and congruences, i.e,RC

(viii) Pick representative for each class in RC

(ix) If x∗i ≡ y∗i for 1 ≤ i ≤ s then F is UNSAT.

(x) If any class in RC contains u∗i and a vertex CONSthen F if unsat.

(xi) Otherwise F is SAT.

29

• ComplexityDetermining satisfiability of a formula of length n isO(nlogn)

• Quantifier free validity problem for TL is decidable(Nelson and Oppen, 1980), (Oppen, 1980)

30

Theory of Arrays, TA

• Function symbols read(a, i) and write(a, i, e)read(a, i) is the result of reading array a at locationiwrite(a, i, e) is a new array b which is the same asa except read(b, i) = ewhere a an array, i an index, e an element

• Examplewrite(a, i, e) = write(b, i, e) ∧read(a, x) 6= read(b, x) ∧ x 6= i

31

• Axioms– ∀a∀i∀e [read(write(a, i, e), i) = e]– ∀a∀i∀j∀e [i 6= j → read(write(a, i, e), j) =read(a, j)]

– ∀a∀b [(∀i (read(a, i) = read(b, i))) → a = b]

32

Extending Congruence Closure to a Sat.procedure for TA

• Consider the theory with symbols read, write, =and 6=– Add uninterpreted function symbols as we are

going to extend congruence closure

• Axioms– ∀a∀i∀e [read(write(a, i, e), i) = e]– ∀a∀i∀j∀e [i 6= j → read(write(a, i, e), j) =read(a, j)]

33

Example

• Consider the formula,F ≡ write(a, i, e) = write(b, i, e) ∧ read(a, x) 6=read(b, x) ∧ x 6= i

• Again like for Lists just doing closure for equivalenceand axioms is not enough.

write write

read read

ia e b

x

• It is closed under equivalences and axioms

• it seems satisfiable

34

Cont.

• Take any interpretation Ψ which satisfies theaxioms.

– Ψ(read(write(a, i, e), x)) = Ψ(read(a, x))– Ψ(read(write(b, i, e), x)) = Ψ(read(b, x))– Ψ(read(a, x)) = Ψ(read(b, x))

• Impossible, contradicts with read(a, x) 6=read(b, x)

write write

read read

ia e b

x

The algorithm is not complete. We need to extend thecongruence closure procedure.

35

Rule 1

• Again like for Lists we look at two possible solutions(i) Extend the set of axioms or(ii) Add more nodes to the graph

• Rule 1– Add the axiom∀b∀b′∀i∀e∀e′ write(b, i, e) = write(b′, i, e′) →e = e

′or

– Whenever write(b, i, e) is represented,read(write(b, i, e), i) is also represented

write

read

write

i e i’b’ e’b

36

Rule 2

• Add the axiomi 6= j ∧ i 6= j′ → read(write(a, j, e), i) =read(write(a, j

′, e′), i) or

• Whenever read(write(a, j, e), i) is represented andi∗ 6= j∗ then read(a, i) is also represented.

read

read read

write write

ja e j’ e’

i

37

Rule 3

• Add the axiomwrite(b, j, e) = write(b

′, j′, e′) ∧ i 6= j ∧ i 6= j′ →

read(b, i) = read(b′, i) or

• Whenever write(b, j, e) and read(b, i) arerepresented and i∗ 6= j∗ then read(write(b, j, e), i)is also represented.

read

readread

write write

b j e j’ e’b’

i

38

Remark

We add three rules to congruence closure procedure tomake the algorithm complete.

• Validity problem for TA is undecidable(Stump et.al., 2001)

• Quantifier free validity problem for TA is decidable(Downey and Sethi, 1978), (Stump et.al., 2001)

39

Theory of reals, TR

• ΣR = {0, 1,+,−,

Theory of Integers, TZ, (PresburgerArithmetic)

• ΣZ = {0, 1,+,−,

Decision Procedures for theory of Realsand Integers

• If the language is reach enough (has multiplication,has quantifiers), deciding validity of arbitrarymathematical formulas is impossible.

• With a more impoverished language, a theory maybe decidable.

42

Types Quantifier Elimination Methods

• We discuss two quantifier elimination methods

1. Fourier-Motzkin variable elimination (FMVE)algorithm to determine validity of formulas whenvariables are from R (or Q) and

2. Omega test (due to Bill Pugh) when variables arefrom Z (or N)

43

Quantifier Elimination

• FMVE is a quantifier elimination procedure

• If a formula with no free variables has no quantifiers,then it is easy to determine its truth value, e.g,10 > 11 ∨ 3 + 4 < 5× 3− 6

• Quantifier elimination works by taking an inputformula P with n quantifiers and turning it into anequivalent formula P

′with m quantifiers and where

m < n

• Eventually, P ≡ P ′ ≡ . . . ≡ Q and Q has noquantifiers.

• Q will be trivially true or false.

44

Fourier-Motzkin Theorems

• Over R (or Q), with a, b > 0 the following are true.1. (∃x. c ≤ ax ∧ bx ≤ d) ≡ bc ≤ ad2. (∃x. c < ax ∧ bx ≤ d) ≡ bc < ad3. (∃x. c ≤ ax ∧ bx < d) ≡ bc < ad4. (∃x. c < ax ∧ bx < d) ≡ bc < ad

45

Proofs

Proof of 1

• Left ⇒ Right– In all the above equivalences, assume there is anx such that c < ax ∧ bx ≤ d, then by transitivitybc < ad

• Right⇒ Left– In the other direction, bc < ad⇒ (c < ax∧ bx ≤d)

– Assume bc < ad, which implies c < a(db). Take xto be db : c < a(

db), and b(

db) ≤ d

46

Combining many constraints-I

• No lower bound, many upper bounds:(∃x.b1x < d1 ∧ b2x < d2 ∧ . . . ∧ bnx < dn)– Take x to be min(dibi) or min(

dibi

) − 1 to be insafe

• No upper bound but many lower bounds(∃x.b1 < d1x ∧ b2 < d2x ∧ . . . ∧ bn < dnx)– Take x to be max(bidi)

• Above formulas always simplifies to true.

47

Cont.

• (∃x.c ≤ ax ∧ b1x ≤ d1 ∧ b2x ≤ d2) ≡ b1c ≤ ad1 ∧b2c ≤ ad2– From left to right, the fact depends on transitivity– From right to left, take x to be min(d1b1 ,

d2b2

)

• In general with many constraints, combine allpossible lower-upper bound pairsThis can be proved by induction on number ofconstraints

48

Combining many constraints-II

• The core elimination formula is∃x.(∧hch ≤ ahx) ∧ (∧ici < aix)∧ (∧jbjx ≤ dj) ∧ (∧kbkx < dk)≡(∧h,jbjch ≤ ahdj) ∧ (∧h,kbkch < ahdk)∧ (∧i,jbjci < aidj) ∧ (∧i,kbkci < aidk)

49

The Procedure

• The method works by eliminating innermostexistential quantifiers, so universal quantifiers arenormalised with (∀x.P (x)) ≡ ¬(∃x.¬P (x))

• The subformula under the innermost existentialquantifier must be a conjuction of relations:

– For any formula ∃x.P (x), where P (x) isquantifier-free, convert P (x) to disjunctivenormal form:(C11 ∧ . . . ∧ C1n1) ∨ . . . ∨ (Cm1 ∧ . . . ∧ Cmnm)

– Then ∃ must be moved inwards using(∃x. P (x) ∨Q(x)) ≡ (∃xP (x)) ∨ (∃xQ(x))

50

Cont.

• If any of the conjucts does not mention the boundvariable x, move it to the side by applying theequivalence (∃x.P (x) ∧Q) ≡ (∃x.Px) ∧Q

• Convert all the relational operators under eachquantifier so that only < and ≤ remain.a = b ≡ (a ≤ b ∧ b ≤ a), a ≥ b ≡ b ≤ a¬(a ≤ b) ≡ b+ 1 ≤ a

• Isolate x in each conjunct and ensure it has apositive coefficient.

• Apply the elimination theorem above to eachquantifier

51

Example

• ∀x. 20 + x ≤ 0 ⇒ ∃y. 3y + x ≤ 10 ∧ 20 ≤ y − x– (re-arrange)

• ≡ ∀x. 20 + x ≤ 0 ⇒ ∃y. 20 + x ≤ y ∧ 3y ≤ 10− x– (eliminate y)

• ≡ ∀x. 20 + x ≤ 0 ⇒ 60 + 3x ≤ 10− x– (re-arrange)

• ≡ ∀x. 20 + x ≤ 0 ⇒ 4x+ 50 ≤ 0– (normalise universal quantifier)

• ≡ ¬∃x. 20 + x ≤ 0 ∧ 0 ≤ 4x+ 50– (re-arrange and eliminate x)

• ≡ ¬∃x. x ≤ −20 ∧ −50 ≤ 4x (eliminate x)

• ≡ ¬(−50 < −80) ≡ >

52

Efficiency

• With n constraints initially, evenly divided betweenupper and lower bounds, eliminating an existential

quantifier generates n2

4 new constraints.

• With k quantifiers to eliminate, we might end with(n

2

4 )k

constraints (quadratic increase).

• If dealing with alternating quantifiers, we have torepeatedly convert to DNF. Bad !

53

Application-I (Constraint Satisfaction)

• The algorithm can return satisfying assignment (ifexists) for a purely existential problem

– Let the Input formula be ∃~x.P where P includesno other quantifiers

– Reduce the problem to one variable– A satisfying value for this variable is anything

between greatest of its lower bounds (glb) andthe least of its upper bounds (lub).

– Substitute this value in the formula in the previousstage.

– Continue this process to get the satisfying valuesfor all the variables

54

Fourier-Motzkin for Z

• Central theorem for eliminating existentialquantifiers is false :

– ∃x. Z.5 ≤ 2x < 6 6≡ 10 ≤ 10– ∃x. Z.5 ≤ 2x < 6≡ ∃x. 5 ≤ 2x ∧ 2x+ 1 ≤ 6≡ ∃x. 5 ≤ 2x ∧ 2x ≤ 56≡ 10 ≤ 10

55

Phase-I

• But one direction still works∃x : Z. (c ≤ ax ∧ bx ≤ d) ⇒ bc ≤ ad

• This is Phase− 1 of Omega test when there are noalternating quantifiers

• Thus we get an incomplete semi procedure foruniversal formulas over Z :

• Compute negation : (∀x.P (x)) ≡ ¬(∃x¬P (x))

• Compute consequences :If (∃x.¬P (x)) ⇒ ⊥ then ¬(∃x.¬P (x)) ≡ >,so (∀x.P (x)) ≡ >

• Phase1 shows an existential formula to be false

56

Real Shadow and Exact Shadow

• Given ∃x.(∧iai ≤ cix) ∧ (∧jdjx ≤ bj)

• The formula ∧i,jaidj ≤ bjci which is implied by theabove formula is known as the real shadow

• If all of the ci or all of the dj are equal to 1, thenthe real shadow is exact

• If the shadow is exact, then the formula can be usedas an equivalence

• When c = 1 or d = 1, the core theorem(∃x : Z. a ≤ cx ∧ dx ≤ b) ≡ ad ≤ bc is validbecause

– ⇒: Transitivity still holds– ⇐: Take x = b if d = 1, x = a if c = 1

57

Dark Shadows

• The formula∧i,j(ci − 1)(dj − 1) ≤ bjci − aidjis known as the dark shadow.(If all ci or all dj are 1 then this is the same as thereal shadow (exact).

• The dark shadow tests for satisfiability, because(c− 1)(d− 1) ≤ bc− ad⇒ (∃x. a ≤ cx ∧ dx ≤ b)

• This is the phase 2 of Omega test.

• Phase 2 shows an existential formula to be true

• The real shadow provides a test for unsatisfiability

58

Omega Test Phases 1 and 2

To find the validity of ∃~x. P (~x)

• If input is exact for one of ~x, then eliminate thisvariable(∃~x. P (~x)) ≡ (∃~x′. P (~x′))where ~x′ contains one variable less than ~x

• Otherwise calculate the real shadow R :(∃~x. P (~x)) ⇒ Rso if R = ⊥ then input formula is not valid.

• Otherwise calculate dark shadow D :D ⇒ (∃~x. P (~x))so if D = >, then input formula is valid

59

Remark

• Purely existential formulas are often– Proved false by their real shadow (Phase 1)– Proved true by their dark shadow (Phase 2)

• The combination of the two is not complete

• Some purely existential formulas might not bedecided by both phases

• It can not handle alternating quantifier problemsalso

60

Splinters

• But in worst cases (not decided by both phases) ,the main theorem (due to Bill Pugh) is neededLet m be the maximum of all the djs. Then

(∃x. (∧iai ≤ cix) ∧ (∧jdjx ≤ bj)) ≡(∧i,j(ci − 1)(dj − 1) ≤ bjci − aidj) ∨∨i ∨b

mci−ci−mm c

k=0 (∃x. (∧iai ≤ cix) ∧ (∧jdjx ≤ bj) ∧(cix = ai + k))

• (∧i,j(ci − 1)(dj − 1) ≤ bjci − aidj) is dark shadow

• (∃x. (∧iai ≤ cix) ∧ (∧jdjx ≤ bj) ∧ (cix = ai + k))is the splinter

61

In the above formula

• A splinter represents a smaller problem than theoriginal because the extra equality allows x to beeliminated

• When the quantifiers alternate and there is no exactshadow, the main theorem is used as an equivalenceand splinters can not be avoided.

• Splinters must also be checked if neither real nordark shadows decide an input formula.

62

Cooper’s Algorithm

A non-Fourier-Motzkin alternative :

(i) Cooper’s algorithm is a decision procedure for(integer) Presburger arithmetic

(ii) It is also a quantifier elimination procedure,which also works from the inside out, eliminatingexistentials

(iii) Its big advantage is that it does not need tonormalise the input formulas to DNF

• The correctness of the Omega test and Cooper’salgorithm are alterantive proofs of Presburger’s 1929result that Presburger arithmetic is decidable.

• Fourier-Motzkin methods are very simple techniquesfor solving problems in R, Q, Z, N.

63

References

[1] Ackermann, W., Solvable Cases of the DecisionProblem, North-Holland, Amsterdam, 1954.

[2] Nelson, G., and Oppen, D. C., Fast DecisionProcedure Based on Congruence Closure, Journalof the ACM, 27(2), 356-364, 1980.

[3] Pugh, W., The Omega Test: a fast and practicalinteger programming algorithm for dependenceanalysis, Communications of the ACM, Vol. 35(8),102-114, 1992.

[4] Oppen, D. C., Reasoning about Recursively DefinedData Structures, Journal of the ACM, 27(3), 403-411, 1980.

[5] Shostak, R. E., An Algorithm for Reasoning AboutEquality, Communications of the ACM, Vol. 21(7),583-585, 1978.

64

[6] Stump, A., Barret, C. W., Dill, D. L., and Levitt,J., A Decision Procedure for an Extensional Theoryof Arrays, In Sixteenth Annual IEEE Symposium onLogic in Computer Science, 29-37, IEEE ComputerSociety, 2001.

[7] Cooper, D. C., Theorem Proving in Arithmeticwithout Multiplication, In Mach. Intell, 7, B Meltzerand D. Michie, Eds, American Elsevier, New York,91-99, 1972.

[8] Kreisel, G. and Krivine, J. L., Elementsof Mathematical Logic, Studies in Logic andthe Foundations of Mathematics, North-HollandPublishing Company, 1967.

[9] Lassez, J-L., and Mahler, M. J., On Fourier’sAlgorithm for Linear Constraints, Journal ofAutomated Reasoning Vol. 9(3), 373–379, 1992.

65