Reading and Writing Mathematical Proofs
Spring 2015
Lecture 4: Beyond Basic Induction
Previously on Reading and Writing
Mathematical Proofs
Proving Correctness of Algorithms
Hoare Logic
Hoare Logic Formal system for logical reasoning about computer programs
Hoare triple
{P} C {Q}
Hoare logic contains rules to determine if Hoare triple is correct If P holds, then after running C, Q holds
Pre- and postcondition are statements about variables
precondition postcondition
command(s)
Hoare Triples
Maximum(A, n)// Algorithm that computes sum of integers in A[1..n]1. {A contains n integers} 2. r = 03. {r = 0}4. for i = 1 to n5. do {r = sum of elements in A[1..i-1]}6. r = r + A[i]7. {r = sum of elements in A[1..i]}8. {r = sum of elements in A[1..n]}9. return r
Goal is to prove Hoare triple {P} C {Q} where C is whole program We have inference rules for single commands Must “break down” Hoare triple into components
loop invariant
While Rule
While Rule
P ⇒ S , {S ⋀ B} C {S} , S ⋀ ¬B ⇒ Q{P} while B do C {Q}
We already know how to prove loops S is the invariant P ⇒ S is the initialization {S ⋀ B} C {S} is the maintenance S ⋀ ¬B ⇒ Q is the termination
It is hard to come up with a good invariant Therefore you must always prove it in Data Structures!
Summary
Hoare logic Formal system for proving algorithms Basically defines the “rules of the game”
Proofs in Data Structures No Hoare logic! (only in the background) Assignments: generally without proof If-statements: prove using case distinction Loops: prove using loop invariant
Always make the distinction between “what the code does” and “what it is supposed to do”! The goal is to prove that these two things are the same
Proving Steps
Steps of Proving
1. Figure out what needs to be shown When is a proof complete? What are the proof requirements?
2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques
3. Write down the proof As clearly as possible With the reader in mind
Hoare logic
No Hoare logic
Nested Loops
How to prove nested loops?
1. {P1}
2. for i = 1 to n
3. do {S1}
4. …..
5. {P2}
6. for j = i+1 to n
7. do {S2}
8. ….
9. {Q2}
10. {Q1}
{P1} for i = 1 to n do OLB {Q1}
P1 ⋀ i=1 ⇒ S1 S1 ⋀ i=n+1 ⇒ Q1
{S1 ⋀ i≤n} OLB {S1[i+1/i]}
{S1 ⋀ i≤n} … {P2}
{P2} for j = i+1 to n do … {Q2}
Q2 ⇒ S1[i+1/i]
P2 ⋀ j=i+1 ⇒ S2 S2 ⋀ j=n+1 ⇒ Q2
{S2 ⋀ j≤n} … {S2[j+1/j]}
Beyond Basic Induction
Today…
Recursion
But first…
Function Calls
How to prove a function call?
1. {P}
2. EpicFunction(x)
3. {Q}
Function should have specification: pre- and postcondition If precondition is met, then postcondition will follow Function is proved separately
What if the function call is recursive?
Then we need a different proving technique…
Recursion
EpicFunc(x)
1. {P}
2. if x = 1
3. then return 1
4. else r = EpicFunc(x-1)
5. r = r + EpicFunc(⌊x/2⌋)6. {Q}7. return r
Cannot prove what EpicFunc does using what EpicFunc does Or can we…?
Recursive calls must have “smaller” input We can use strong induction!
No idea what this does!Let’s try something else
Example
BinarySearch(A, i, j, x)
// Returns true iff A[i…j] contains x
1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j]}2. if i + 1 = j
3. then return (A[i] = x)
4. h = (i + j)/2
5. if A[h] ≤ x6. then return BinarySearch(A, h, j, x)7. else return BinarySearch(A, i, h, x)
Strong induction Base case(s): when no recursive calls are made Induction step: the rest… Induction on what?
Must be “smaller” input!
ExampleTheorem
If A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j], then BinarySearch(A, i, j, x) returns true iff A[i…j] contains x
Proof
We use strong induction on |j – i|.
Base case (|j – i| = 1):Then, since A[j] > x, only A[i] can contain x. This is correctly checked by the algorithm.
Step (|j – i| ≥ 2):First note that i < h < j, so 1 ≤ |j – h| < |j – i| and 1 ≤ |h – i| < |j – i|. We consider two cases:
Case (1): A[h] ≤ x
Since A is sorted, x cannot be in A[i…h-1] and must be in A[h…j]. This is checked by the recursive call. As required, A[h] ≤ x.
Case (2): A[h] > x
Since A is sorted, x cannot be in A[h+1…j] and must be in A[i…h]….
Recursion
Notes on recursion
Always make sure recursive calls are valid Input should satisfy requirements Input should be “smaller” than original input Must eventually reach a base case
What does “smaller” input mean? You get to define it! But must satisfy certain requirements…
Induction
A more general approach…
Induction
Different types of induction
Mathematical induction P(1) P(n) ⇒ P(n+1)
Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)
Structural induction ?
Well-founded induction ?
For natural numbers
For other sets
Induction
Mathematical Induction
Prove something for all positive integers: ∀n[n ϵ ℕ: P(n)]
What if we want to use a different set: ∀x[x ϵ S: P(x)] Cannot use standard induction: P(x) ⇒ P(x+1) What to do?
Examples Prove for all strings that … Prove for all rooted binary trees that … Prove for all graphs that … Prove for all polygons that …
Set definitions
Defining Infinite Sets
Using properties: Set of rationals = {x | ∃p,q ϵ ℤ[q ≠ 0 and qx = p]}
Set of primes = {p | ¬∃d ϵ ℤ[1 < d < p and p is multiple of d]}
Set of squares = {x2 | x ϵ ℤ}
Inductive (or recursive) definition: Natural numbers ℕ:
1) 1 ϵ ℕ2) If n ϵ ℕ, then n + 1 ϵ ℕ
Full binary trees T:1) ϵ T
2) If x ϵ T and y ϵ T, then ϵ Tx y
Useful for induction!
Examples
1. Set of positive even numbers E 2 ϵ E If n ϵ E, then n + 2 ϵ E
2. Set of (non-empty) binary strings B 0 ϵ B, 1 ϵ B If X ϵ B, then 0X ϵ B and 1X ϵ B
3. Set of powers of 3: Q 1 ϵ Q If p ϵ Q, then 3p ϵ Q
4. Set of arithmetic expressions A n ϵ A for all n ϵ ℕ If e1, e2 ϵ A, then –e1, (e1), e1 + e2, e1 – e2, e1 * e2, e1 / e2 ϵ A
5. Set of prime numbers P Don’t know…
Structural Induction
Full binary trees T
1) ϵ T
2) If x ϵ T and y ϵ T, then ϵ T
Structural Induction on T Base case
Prove property for a single node Induction step
Prove property for
Can use induction hypothesis on x and y
x y
x y
Example
Theorem
A full binary tree with n nodes has (n+1)/2 leaves
Proof
We use structural induction on the set of full binary trees with the inductive rule on the previous slide.
Base case (rule (1)):For a single node n = 1 and there is (1+1)/2 = 1 leaf.
Step (rule (2)):Suppose that the subtrees x and y, with a and b nodes, have (a+1)/2 and (b+1)/2 leaves, respectively (IH). We need to show that the tree T with n nodes formed by adding a root above x and y has (n+1)/2 leaves.
A leaf of T is either a leaf of x or a leaf of y. By IH, the number of leaves of T is then (a+1)/2 + (b+1)/2 = (a+b+2)/2. Since n = a + b + 1, we get that (n+1)/2 = (a+b+2)/2, as required.
Example
Theorem
A full binary tree with n nodes has (n+1)/2 leaves
Proof
We use structural induction on the set of full binary trees with the inductive rule on the previous slide.
Base case (rule (1)):For a single node n = 1 and there is (1+1)/2 = 1 leaf.
Step (rule (2)):Consider a full binary tree T with subtrees x and y.
Let x and y have a and b nodes, respectively, such that n = a + b + 1.
A leaf of T is either a leaf of x or a leaf of y. By the IH, we get that T has (a+1)/2 + (b+1)/2 = (a+b+2)/2 = (n+1)/2 leaves.
Isn’t this the same as strong induction?
Practice
Fancy Sequences S
1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ
ϵ S
What does this mean? S contains sequences of integers Any sequence of 1 integer is in S If x ϵ ℕ is not in sequences Y, Z ϵ S, then YxZ is also in S
Examples 12345 ϵ S 121423 ϵ S 12334 ∉ S 12121 ∉ S
Practice
Fancy Sequences S
1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ
ϵ S
TheoremEvery non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely
PracticeFancy Sequences S
1) x ϵ S for all x ϵ ℕ2) If Y, Z ϵ S and x ϵ ℕ such that x ∉ Y and x ∉ Z, then YxZ ϵ S
TheoremEvery non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely
ProofWe use structural induction on fancy sequences with the above rule.Base case (rule (1)):
The (sub)sequence contains only one number, so it must be unique.
Step (rule (2)):We perform a case distinction based on the subsequence:Case 1 (subsequence contains x): Since x is not in Y or Z, x is unique.Case 2 (subsequence contained in Y): By the IH on Y, the result holds.Case 3 (subsequence contained in Z): By the IH on Z, the result holds.
Induction
Different types of induction
Mathematical induction P(1) P(n) ⇒ P(n+1)
Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)
Structural induction Induction using inductive definition of set
Well-founded induction ?
Binary Search (again)
BinarySearch(A, i, j, x)
// Returns true iff A[i…j] contains x
1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j]}2. if i + 1 = j
3. then return (A[i] = x)
4. h = (i + j)/2
5. if A[h] ≤ x6. then return BinarySearch(A, h, j, x)7. else return BinarySearch(A, i, h, x)
Smaller input Size of input defined as |j – i| ⇒ strong induction on |j –
i| In general: Order on set of possible inputs (A, i, j, x) ≺ (A’, i’, j’, x’) if |j – i| < |j’ – i’|
Must be “smaller” input!
Partial order
Strict partial order relation ≺ on set S Binary relation: x ≺ y for certain pairs x, y ϵ S Anti-reflexive: x ⊀ x Anti-symmetric: if x ≺ y, then y ⊀ x Transitive: If x ≺ y and y ≺ z, then x ≺ z
Examples For x, y ϵ ℤ: x ≺ y iff x < y For sets X, Y: X ≺ Y iff X ⊂ Y ({1,3} ≺ {1,2,3,4}) For strings S1, S2: S1 ≺ S2 iff S1 is substring of S2 (“ab”
≺ “cab”) For trees T1, T2: T1 ≺ T2 iff T1 is substring of T2
Partial order
Does (strong) induction work for any partial order ≺ ? Not exactly…
Theorem
For all x ϵ ℤ it holds that x = x + 1Proof
By induction on x:
We apply IH to x – 1, so that x – 1 = x. (Note that x – 1 < x)
By adding 1 to both sides we obtain that x = x + 1.
We need base cases! The partial order ≺ must have minimal elements Minimal elements are base cases
Well-founded Relation
Well-founded relation ≺ on S Every non-empty subset X ⊆ S must have a minimal
element Minimal element m ϵ X: for all x ϵ X it holds that x ⊀ m S contains no infinite descending chains: a ≻ b ≻ c ≻ ….
Well-founded induction on S First need well-founded (partial) order ≺ on S Base case(s): Minimal elements of S Induction step: If P(x) for all x ≺ y, then P(y) (for all y ϵ S)
Practice
Which of these partial orders are well-founded?
For x, y ϵ ℤ: x ≺ y iff x < y
For x, y ϵ ℤ: x ≺ y iff |x| < |y|
For x, y ϵ ℕ: x ≺ y iff y is a multiple of x and x ≠ y
For rational numbers x, y ϵ ℚ: x ≺ y iff x < y
For strings S1, S2: S1 ≺ S2 iff S1 lexicographically before S2
½ > ⅓ > ¼ > ⅕ > …
“b” ≻ “ab” ≻ “aab” ≻ “aaab” ≻ …
Practice
Ackermann(m, n)
1. if m = 0
2. then return n+1
3. else if n = 0
4. then return Ackermann(m – 1, 1)
5. else return Ackermann(m – 1, Ackermann(m, n – 1))
A recursive function terminates on all input if and only if there exists a well-founded order ≺ on the set of inputs such that:
“input of recursive call” ≺ “original input”
Does the Ackermann function terminate on all inputs?
(m, n) ≺ (m’, n’) iff m < m’ or m = m’ and n < n’
Practice
T(1, n) = 1
T(m, 1) = 1
T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1 m, n > 1
Theorem
T(m, n) = log(m) log(n) + 1
Practice
Theorem
T(m, n) = log(m) log(n) + 1
Proof
We use induction on (m, n), where (m, n) ≺ (m’, n’) iff m < m’ or n < n’.Base case (m = 1):
T(1, n) = 1 = log(1) log(n) + 1.
Base case (n = 1):T(m, 1) = 1 = log(m) log(1) + 1.
Step (m, n > 1):T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1{definition}T(m, n) = log(m/2) log(n) + log(m) log(n/2) – log(m/2) log(n/2) + 2 {IH}T(m, n) = log(m/2) (log(n) – log(n/2)) + log(m) log(n/2) + 2T(m, n) = log(m/2) + 1 + log(m) (log(n) – 1) + 1T(m, n) = log(m) log(n) + 1
Well-founded Induction
Well-founded induction Very general type of (strong) induction … but also very abstract
Main lesson If you can order the elements of a set, you can do induction Induction hypothesis may always be applied to “smaller” elements
Recursion First argue that it terminates Then you can use IH on recursive call to argue correctness
Induction
Different types of induction
Mathematical induction P(1) P(n) ⇒ P(n+1)
Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)
Structural induction Induction using inductive definition of set
Well-founded induction Strong induction for any set Just needs well-founded order…
Summary
Steps of Proving
1. Figure out what needs to be shown When is a proof complete? What are the proof requirements?
2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques
3. Write down the proof As clearly as possible With the reader in mind
“Rules” of proving
“Rules” of proving
“Tools” of proving
hard, creative, fun(?) part