mathematical preliminaries - home pages of all faculty at kfupm
TRANSCRIPT
Lecture 10: Mathematical Preliminaries
Objective:
Reviewing mathematical concepts and tools that are frequently used in the analysis of algorithms.
Lecture # 10 Slide # 1<BUTTONS AREA: Do not write anything here>
• In this and the next two lectures, our main objective is to give a review of a variety of mathematical concepts and tools that are frequently used in the analysis of algorithms.
• Most of these concepts and tools should be familiar to the student. They are
provided here mainly as a reference and for completeness.
ICS-353 Lecture Template
Topics reviewed in this lecture
• Sets • Tuples and Cartesian Products • Relations and Functions • Proof Methods
- Direct Proof - Indirect Proof - Proof by Contradiction - Proof by Counterexample - Mathematical Induction
• Logarithms • Floor and Ceiling Functions • Factorial and Binomial coefficients • The Pigeonhole Principle
Lecture # 1 Slide # 2<BUTTONS AREA: Do not write anything here>
• The topics briefly reviewed in this lecture include the following:
- Sets - Tuples and Cartesian Products - Relations and Functions - Five mathematical Proof methods which include:
1. The Direct Proof method 2. Indirect Proof 3. Proof by Contradiction 4. Proof by Counterexample 5. Mathematical Induction
- Then we will review logarithms - Floor and Ceiling Functions - Factorial and Binomial coefficients, and finally - The Pigeonhole Principle
First we will introduce Sets
ICS-353 Lecture Template
Sets
Why sets are important to the algorithm course?
Definition: A set is a collection of distinct elements, which are called members or elements of the set. This collection of elements is treated as a single object.
Two commonly used methods to represent a set:
1. By listing its elements between a pair of curly braces, or 2. By describing its elements between curly braces
Lecture # 1 <BUTTONS AREA: Do not write anything here>
Why sets are important to the algorithm course? We study set concepts because input of an algorithm may be considered as a set drawn from some particular domain like the set of integers.
We will briefly present basic concepts and a few elementary properties of sets used in the design and analysis of algorithms. But first let us define sets.
Definition: A set is a collection of distinct elements, which are called members or elements of the set. This collection of elements is treated as a single object.
The two commonly used methods to represent a set are:
The first method is by listing list elements between a pair of curly braces. The second method is by describing list elements between curly braces
ICS-353 Lecture Template
Sets (Cont.)
Examples. Example 1:
• The set of integers between 1 and 100 can be represented as {1, 2, 3, ...,
100}.
• This same set can be represented using the second method as S = {x | 1 ≤ x ≤ 100 and x is an integer}.
• The second method is the only way used to describe uncountable set. Example 2:
S = {x | x is a real number and 0 ≤ x ≤ 1}.
Lecture # 1 Slide # 4<BUTTONS AREA: Do not write anything here>
Examples on Sets: Example 1: Using the first method of representation, the set of integers between 1 and 100 can be represented as {1, 2, 3, ..., 100}. On the other hand, this same set can be represented using the second method by specifying some properties as follows: S = {x | 1 ≤ x ≤ 100 and x is an integer}. The expression for S is read “ the set of all elements x such that x is equal or greater than 1 and less than or equal to 100 and x is an integer.” The second method is the only way used to describe uncountable set. Example 2: The set notation S = {x | x is a real number and 0 ≤ x ≤ 1} is used to represent the set of real numbers between 0 and 1.
ICS-353 Lecture Template
Sets (Cont.)
Set Properties • A set is finite if it contains n elements for some constant n > 0, and infinite otherwise.
- Example 1 above represents a finite set.
- Examples of infinite sets: • Set of natural numbers {1, 2, …} • Set of integers • Set of rationals and • Set of reals.
• Infinite set is countable if its elements can be listed as the first element, second element,
and so on; otherwise it is called uncountable. The set of real numbers is uncountable.
Lecture # 1 <BUTTONS AREA: Do not write anything here>
Elementary properties of sets • A set is said to be finite if it contains n elements for some constant n > 0, and infinite
otherwise. - The set in Example 1 above is a finite set. - Examples of infinite sets are:
• Set of natural numbers {1, 2, …} • Set of integers • Set of rationals and • Set of reals.
• An infinite set is called countable if its elements can be listed as the first element,
second element, and so on; otherwise it is called uncountable. For example, the set of real numbers is uncountable.
ICS-353 Lecture Template
Sets (Cont.)
Set properties (Cont.)
• The empty set is denoted by {} or by φ, and it is a subset of every set. • The cardinality of a finite set A denoted by |A|, is the number of elements in A.
• x ∈ A indicates that x is a member of A. • A set B is a subset of a set A, denoted by B ⊆ A, if each element of B is an element
of A. A is said to be a superset of the set B.
• B ⊂ A denotes that B is a proper subset of A. That is, in addition to B ⊆ A, B ≠ A.
• For any set A, A ⊆ A, and φ ⊆ A.
• If A and B are two sets, then A ⊆ B and B ⊆ A, means A = B.
• The Union of two sets A and B, denoted by A U B, is the set {x | x ∈ A or x ∈ B}
Lecture # 1 Slide # 6<BUTTONS AREA: Do not write anything here>
• The empty set is denoted by {} or by φ, and it is a subset of every set. • The cardinality of a finite set A denoted by |A|, is the number of elements in A.
• x ∈ A indicates that x is a member of A. • A set B is a subset of a set A, denoted by B ⊆ A, if each element of B is an
element of A. A is said to be a superset of the set B.
• B ⊂ A denotes that B is a proper subset of A. That is, in addition to B ⊆ A, B ≠ A.
• For any set A, A ⊆ A, and φ ⊆ A.
• If A and B are two sets, then A ⊆ B and B ⊆ A, means A = B.
• The Union of two sets A and B, denoted by A U B, is the set {x | x ∈ A or x ∈ B}
ICS-353 Lecture Template
Sets (Cont.)
Set properties (Cont.)
• The intersection of two sets A and B, denoted by A ∩ B, is the set {x | x ∈ A and x ∈ B}.
• The difference of a set A from a set B, denoted by A - B, is the set {x | x ∈ A and x ∉ B}.
• The complement of a set A, denoted by A , is defined as U - A, where U is the universal set containing A, which is usually understood from the context.
• If A, B and C are sets, then A U ( B U C) = (A U B) U C, and A ∩ (B ∩ C) = (A ∩ B) ∩ C.
• Two sets A and B are said to be disjoint if A ∩ B = φ. • The power set of a set A, denoted by P(A), or 2A is the set of all subsets of A. • If |A| = n, then |P (A) | = 2 n. Note that φ ∈P(A) and A ∈P(A). • To describe the number of distinct subsets of cardinality k that a finite set of n
elements have, we use the special notation , read as “number of combinations of
n items taken k at a time, or “n choose k.” Also, the notation C (n, k) is sometimes used. These quantities are called binomial coefficients, as will be discussed later.
kn
Lecture # 10 Slide # 7<BUTTONS AREA: Do not write anything here>
• The intersection of two sets A and B, denoted by A ∩ B, is the set {x | x ∈ A and
x ∈ B}. • The difference of a set A from a set B, denoted by A - B, is the set {x | x ∈ A and
x ∉ B}. • The complement of a set A, denoted by A , is defined as U - A, where U is the
universal set containing A, which is usually understood from the context. • If A, B and C are sets, then A U ( B U C) = (A U B) U C, and A ∩ (B ∩ C) = (A ∩
B) ∩ C. • Two sets A and B are said to be disjoint if A ∩ B = φ. • The power set of a set A, denoted by P(A), or 2A is the set of all subsets of A. • If |A| = n, then |P (A) | = 2 n. Note that φ ∈P(A) and A ∈P(A). • To describe the number of distinct subsets of cardinality k that a finite set of n
elements have, we use the special notation , read as “number of combinations
of n items taken k at a time, or “n choose k.” Also, the notation C (n, k) is sometimes used. These quantities are called binomial coefficients, as will be discussed later.
kn
ICS-353 Lecture Template
Tuples and the Cartesian Products
• An ordered n-tuple is an ordered collection (a1, a2, …, an) that has a1 as its first element, a2 as its second element,…, and an as its nth element.
• A tuple often do not have the same type. • Short tuples have special names: ordered pairs refer for 2-tuples, triples for 3-tuples,
quadruple for 4-tuples, quintuple for 5-tuples, and so on.
• The Cartesian (or cross) product of two sets A and B, denoted by A * B, is the set of all ordered pairs (a, b) where a ∈ A and b ∈ B. In set notation: A * B = {(a, b) | a ∈A and b ∈B}.
• In general, the Cartesian product of n sets is: A1 * A2 * …* An = {(a1, a2,…, an) | ai ∈Ai, 1 ≤ i ≤ n }.
Lecture # 1 Slide # 8<BUTTONS AREA: Do not write anything here>
Tuples and the Cartesian Products
• An ordered n-tuple is an ordered collection (a1, a2, …, an) that has a1 as its first element, a2 as its second element,…, and an as its nth element.
• A tuple often do not have the same type. • Short tuples have special names: ordered pairs refer for 2-tuples, triples for 3-
tuples, quadruple for 4-tuples, quintuple for 5-tuples, and so on.
• The Cartesian (or cross) product of two sets A and B, denoted by A * B, is the set of all ordered pairs (a, b) where a ∈ A and b ∈ B. In set notation: A * B = {(a, b) | a ∈A and b ∈B}.
• In general, the Cartesian product of n sets is: A1 * A2 * …* An = {(a1, a2,…, an) | ai ∈Ai, 1 ≤ i ≤ n }.
ICS-353 Lecture Template
Relations and Functions
• Why we study functions and relations? • Elementary properties of relations and functions.
- Let A and B be two nonempty sets, a binary relation, or simply a relation, R from A to B is a set of ordered pairs (a, b) where a ∈A and b∈B. That is R ⊆ A * B.
- If A = B, we say that R is a relation on the set A. - The domain of R written as Dom (R) = {a | for some b ∈B, (a, b) ∈ R}. - The range of R written as Ran (R) = {b | for some a ∈A, (a, b) ∈ R}.
Examples on domain and range S1 = {(2, 5), (3, 3)}, S2 = {(x, y) | x, y are positive integers and x ≤ y} and S3 = {(x, y) | x, y are real numbers and x2 + y2 ≤ 1} then, Dom (S1) = {2, 3}; Ran (S1) = {5, 3); Dom (S2) = Ran (S2) is the set of natural numbers, and Dom (S3) = Ran (S3) is the set of real numbers in the interval [-1 .. 1].
Lecture # 1 Slide # 9<BUTTONS AREA: Do not write anything here>
Relations and Functions • Why we study functions and relations? We study relations and functions because
an algorithm can be considered as a function (or constrained relation) that maps each possible input to a specific output.
• Elementary properties of relations and functions used in the design and analysis of algorithms: - Let A and B be two nonempty sets, a binary relation, or simply a relation,
R from A to B is a set of ordered pairs (a, b) where a ∈A and b ∈B. That is R ⊆ A * B.
- If A = B, we say that R is a relation on the set A. - The domain of R written as Dom (R) = {a | for some b ∈B, (a, b) ∈ R}. - The range of R written as Ran (R) = {b | for some a ∈A, (a, b) ∈ R}.
Example on domain and range Assume the three sets S1 = {(2, 5), (3, 3)}, S2 = {(x, y) | x, y are positive integers and x ≤ y} and S3 = {(x, y) | x, y are real numbers and x2 + y2 ≤ 1} then, Dom (S1) = {2, 3}; Ran (S1) = {5, 3); Dom (S2) = Ran (S2) is the set of natural numbers, and Dom (S3) = Ran (S3) is the set of real numbers in the interval [-1 .. 1].
ICS-353 Lecture Template
Relations and Functions (Cont.)
• Elementary properties of relations and functions (Cont.) - Let R be a relation on a set A: then R is said to be: reflexive if (a, a) ∈ R ∀ a ∈ A. symmetric if (a, b) ∈ R implies (b, a) ∈R. transitive if (a, b) ∈ R and (b, c) ∈ R implies (a, c) ∈ R. irreflexive if (a, a) ∉ R ∀ a ∈ A asymmetric if (a, b) ∈ R implies (b, a) ∉R antisymmetric if (a, b) ∈ R and (b, a) ∈ R implies a = b.
- A relation that is reflexive, antisymmetric and transitive is called a partial
order. Example on a partial order relation:
Let S1 = {(x, y)| x, y are positive integers and x divides y} S2 = {(x, y) | x, y are integers and x ≤ y} then:
both S1 and S2 are reflexive, antisymmetric and transitive and hence both are partial order.
Lecture # 1 Slide # 10<BUTTONS AREA: Do not write anything here>
• Elementary properties of relations and functions (Cont.) - Let R be a relation on a set A: then R is said to be: reflexive if (a, a) ∈ R ∀ a ∈ A. symmetric if (a, b) ∈ R implies (b, a) ∈R. transitive if (a, b) ∈ R and (b, c) ∈ R implies (a, c) ∈ R. irreflexive if (a, a) ∉ R ∀ a ∈ A asymmetric if (a, b) ∈ R implies (b, a) ∉R antisymmetric if (a, b) ∈ R and (b, a) ∈ R implies a = b.
- A relation that is reflexive, antisymmetric and transitive is called a partial
order. Example on a partial order relation:
Let S1 = {(x, y)| x, y are positive integers and x divides y} S2 = {(x, y) | x, y are integers and x ≤ y} then:
both S1 and S2 are reflexive, antisymmetric and transitive and hence both are partial order.
ICS-353 Lecture Template
Relations and Functions (Cont.)
Equivalence Relations • It is a relation that is, reflexive, transitive and symmetric. • If R is a relation on a set A, R partitions A into equivalence classes such that:
- For any Ci, 1 ≤ i ≤ k, if x ∈ Ci and y ∈ Ci, then (x, y) ∈ R. - If x ∈ Ci and y ∈ Ci and i ≠ j, then (x, y) ∉ R.
Example: x ≡ y (mod n) if x - y = k * n for some integer k. That is, x ≡ y (mod n) if both x and y leave the same remainder when divided by n. For instance: 13 ≡ 8 (mod 5) If we define the relation: R = {(x, y) | x, y are integers and x ≡ y (mod n)}, then R is an equivalence relation. R partitions the set of integers into n classes: C0, C1, …, Cn-1 such that x ∈ Ci and y ∈ Ci if and only if x ≡ y (mod n).
Lecture # 1 Slide # 11<BUTTONS AREA: Do not write anything here>
Equivalence Relations • An equivalence relation is a relation that is, reflexive, transitive and symmetric. • If R is a relation on a set A, R partitions A into equivalence classes C1, C2,…, Ck
such that any two elements in one equivalence class are related by R. That is: - For any Ci, 1 ≤ i ≤ k, if x ∈ Ci and y ∈ Ci, then (x, y) ∈ R. - If x ∈ Ci and y ∈ Ci and i ≠ j, then (x, y) ∉ R.
Example on Equivalence relation: Let x and y be two integers, and let n be a positive integer, then x and y are said to be congruent modulo n, denoted by x ≡ y (mod n) if x - y = k * n for some integer k. Thus we say that x and y are congruent modulo n if both x and y leave the same remainder when divided by n. For instance: 13 ≡ 8 (mod 5) as 13 – 8 = 1 * 5, or both 13 and 8 both have the same remainder (=3) when divided by 5. If we define the relation: R = {(x, y) | x, y are integers and x ≡ y (mod n)}, then R is an equivalence relation. R partitions the set of integers into n classes: C0, C1, …, Cn-1 such that x ∈ Ci and y ∈ Ci if and only if x ≡ y (mod n).
ICS-353 Lecture Template
Relations and Functions (Cont.)
Functions:
• Definition • We write this as f (x) = y
Examples: 1. The relation {(1, 2), (3, 4), (2, 4) is a function, 2. The relation {(1, 2), (1, 4) is not since for the element 1 in the domain there is more
than on element in the range. 3. The relation {(x, y)} | x, y are positive integers and x = y3} is a function 4. The relation {(x, y) | x is a positive integer, y is integer and x = y2} is not a function. • Important properties of functions
Lecture # 1 Slide # 12<BUTTONS AREA: Do not write anything here>
Functions: • A function f is a (binary) relation such that for every element x ∈ Dom (f) there
exists one element y ∈ Ran (f) with (x, y) ∈ f. • Usually we write this as f (x) = y instead of (x, y) ∈ f and says that y is the value
or image of f at x. Examples:
1. The relation {(1, 2), (3, 4), (2, 4) is a function, 2. The relation {(1, 2), (1, 4) is not since for the element 1 in the domain there is
more than on element in the range. 3. The relation {(x, y)} | x, y are positive integers and x = y3} is a function 4. The relation {(x, y) | x is a positive integer, y is integer and x = y2} is not a
function. • Properties of functions
- Let f be a function with Dom (f)=A and Ran (f) ⊆ B for nonempty sets A and B: We say that f is one to one if f (x) = f (y) implies x = y, that is if for no different elements x and y in A, f (x)= f (y).
- f is said to be onto B if Ran (f) = B - f is said to be one to one correspondence between A and B (also called
bijection) if it is both one to one and onto B.
ICS-353 Lecture Template
Proof Techniques
1. Direct Proof
2. Indirect Proof
3. Proof by Contradiction
4. Proof by Counterexample
5. Proof by Mathematical Induction
Lecture # 1 Slide # 13<BUTTONS AREA: Do not write anything here>
Proof Techniques Mathematical proofs is considered as an essential component and prerequisite to algorithm design and analysis. We will give here a brief review of five proof methods that are commonly used in algorithm design and analysis.
1. Direct Proof 2. Indirect Proof
3. Proof by Contradiction
4. Proof by Counterexample
5. Proof by Mathematical Induction
ICS-353 Lecture Template
Notations used in mathematical proofs
• A proposition or assertion. • The negation symbol “¬”. • The implication symbol “ →”.
• The “if and only if” symbol “↔”.
• Thus:
- “P → Q” stands for “P implies Q” or “if P then Q”. - The statement “P ↔ Q”, stands for: “P if and only if Q”. It consists of two parts
(i.e., two implications) each of which should be proven separately: The if part (implication): “P → Q”.
The only if part (implication): “Q → P”.
Lecture # 1 Slide # 14<BUTTONS AREA: Do not write anything here>
Notations used in mathematical proofs
The following notation is used in mathematical proof. • A statement that can be either true or false but not both is called a proposition or
assertion. • The symbol “¬” is the negation symbol. • The symbol “ →” is the implication symbol, and “↔” is the “if and only if”
symbol. Thus: - “P → Q” stands for “P implies Q” or “if P then Q”. - The statement “P ↔ Q”, stands for: “P if and only if Q”. That is: “P is true if
and only if Q is true”. It consists of two parts (two implications) each of which should be proven separately: The if part (implication): “P → Q”.
The only if part (implication): “Q → P”.
ICS-353 Lecture Template
Direct proof method
We want to prove that if P then Q, that is: “P → Q”. We start the proof by assuming that P is true, and then deduce the truth of Q from the truth of P. Example: If N is an even integer, then prove that N is also even integer. 2
Solution:
Since N is even, N = 2k for some integer k.
So, N2 = (2K)2 = 4K2 = 2 (2K 2).
It follows that N 2 is an even integer.
Lecture # 1 Slide # 15<BUTTONS AREA: Do not write anything here>
Direct Proof method We want to prove that if P then Q, that is: “P → Q” We start the proof by assuming that P is true, and then deduce the truth of Q from the truth of P. Example: Use the direct proof method to prove that if N is an even integer, then provthat N 2 is also even integer. Solution: Since N is even, N = 2k for some integer k. So, N2 = (2K)2 = 4K2 = 2 (2K 2).
It follows that N 2 is an even integer.
ICS-353 Lecture Template
Indirect proof method
Indirect proof • “if not Q then not P” • “P → Q” is logically equivalent to “if not Q then not P “ • The statement “if it is raining then it is cloudy” is equivalent to the statement “if it is not
cloudy then it is not raining”. Example: Prove the assertion that “if n2 is an even integer, then n is an even integer” Solution: This assertion is logically equivalent to “if n is an odd integer, then n2 is an
odd integer”. Since n is odd then n = 2k+1 for some integer k. Hence, n2 = (2k + 1) 2 = 4k2 + 4k + 1 = 2(2k2 + 2k) + 1.
Which implies that n2 is an odd integer.
Lecture # 1 Slide # 16<BUTTONS AREA: Do not write anything here>
Indirect proof method It is some times much easier to prove “if not Q then not P” than using a direct proof for the statement “if P then Q”.
“P → Q” is logically equivalent to the contrapositive implication “if not Q then not P ”, which is also written as “¬Q → ¬P ”. For instance, the statement “if it is raining then it is cloudy” is equivalent to the statement “if it is not cloudy then it is not raining”.
Example: Prove the assertion that “if n 2 is an even integer, then n is an even integer” using indirect proof. Solution: This assertion is logically equivalent to “if n is an odd integer, then n2 is an
odd integer” which is proved using the direct method as follows: Since n is odd then n = 2k+1 for some integer k. Hence, n2 = (2k + 1) 2 = 4k2 + 4k + 1 = 2(2k2 + 2k) + 1.
Which implies that n2 is an odd integer.
ICS-353 Lecture Template
Proof by contradiction
• Widely used and extremely powerful. • To prove the statement “if P then Q”:
- Start with the assumption that P is true but Q is false
• Logical reasoning for proof by contradiction.
• Other contradictions may be reached as well.
Lecture # 1 Slide # 17<BUTTONS AREA: Do not write anything here>
Proof by contradiction
• Widely used and extremely powerful. Its power comes from making proofs concise and short as will be seen in the next example.
• To prove the statement “if P then Q”: - start with the assumption that P is true but Q is false: If this leads to a contradiction, it implies that our assumption “Q is false”
must be wrong and hence Q must follow from P.
• Logical reasoning for proof by contradiction: - If we know that P → Q is true and Q is false, then P must be false. - So, if we assume at the beginning that P is true, Q is false, and reach the
conclusion that P is false, then we have that P is both true and false. - But P cannot be both true and false - a contradiction. - We conclude our assumption that Q is false is wrong ⇒ Q must be true.
• Other contradictions may be reached as well:
For instance, after assuming that P is true and Q is false, we may reach the conclusion that, say, 1 = -1.
ICS-353 Lecture Template
Proof by contradiction (Cont.)
Example. Prove the theorem that: If n balls are distributed into m boxes, then one box must contain at least balls. mn /
Solution: If all boxes have less than balls, then the total number of balls is at most mn /
−
1mnm ≤
−
−
+ 11m
mmnm = n + m –1 – m < n
which is a contradiction. This is an excellent example that shows its power in producing very short proofs.
Lecture # 10 Slide # 18<BUTTONS AREA: Do not write anything here>
Example on proof by contradiction: Prove by contradiction the pigeonhole principal theorem that: If n balls are distributed into m boxes, then one box must contain at least balls. mn /
The definition of this principle will be given later in this lecture. Solution: If all boxes have less than balls, then the total number of balls is at most mn /
−
1mnm ≤
−
−
+ 11m
mmnm = n + m –1 – m < n
which is a contradiction. This is an excellent example that shows the power of this proof method in producing very short proofs.
ICS-353 Lecture Template
Proof by counterexample method
General Introduction
Example: Consider the function f (n) = n2 + n + 41 defined on the set of nonnegative
integers:
- f (0) = 41, f (1) =43,…, f (39) = 1601 are all primes. - Assertion: f (n) is always a prime number.
- We use Proof by counterexample, to falsify this assertion:
consider f (40) = 1681 = (41)2 which is not a prime. We conclude
that the assertion is false.
Lecture # 1 Slide # 19<BUTTONS AREA: Do not write anything here>
Proof by counterexample method
General Introduction: - This method is used to show that a proposition that holds true quite often is
not always true. - Also, it is usually used when we are faced with a problem that requires
proving or disproving a given assertion. In this case, we may start by trying to disprove the assertion with a
counterexample. - In algorithm analysis, this technique is frequently used to show that an
algorithm does not always produce a result with certain properties.
Example: Let f (n) = n2 + n + 41 be a function defined on the set of nonnegative integers:
- f (0) = 41, f (1) =43,…, f (39) = 1601 are all primes. - Assertion: f (n) is always a prime number. - We use Proof by counterexample, to falsify this assertion:
consider f (40) = 1681 = (41)2 which is not a prime. We conclude that the assertion is false.
ICS-353 Lecture Template
Proof by mathematical induction
Proof By Mathematical Induction:
• A powerful proof technique. • The truth of a number of statements (propositions) can be inferred from the truth of a
few specific instances.
• Suppose a sequence of statements S1, S2, ..., we want to prove to be true.
• Suppose also that:
1. For some i ≥ 1, we know that S1, S2,…, Si are true. 2. The problem is such that for any n ≥ i, the truths of S1, S2, ... Sn imply the truth
of Sn+1.
• We can then use induction to show that every statement in this sequence is true.
Lecture # 1 Slide # 20<BUTTONS AREA: Do not write anything here>
Proof By Mathematical Induction:
• It is a powerful proof technique for proving that a property holds for a sequence of natural numbers n0, n0 + 1, n0 + 2, …. Typically, n0 is taken to be 0 or 1, but it can be any natural number. Also, the sequence n0, n0 + 1, n0 + 2, …can be any linearly ordered set. However, in the majority of proofs by induction, the set of natural numbers is assumed.
•
• In this technique, the truth of a number of statements (propositions) can be inferred from the truth of a few specific instances.
• Suppose we have a sequence of statements S1, S2, ..., we want to prove to be true.
Suppose also that the following holds: 1. For some i ≥ 1, we know that S1, S2,…, Si are true. 2. The problem is such that for any n ≥ i, the truths of S1, S2, ... Sn imply the
truth of Sn+1.
• We can then use induction to show that every statement in this sequence is true.
ICS-353 Lecture Template
Proof by mathematical induction (Cont.)
Example Prove that a binary tree of height n has at most 2n leaves: Proof: Let L (n) represent the number of leaves of a binary tree of height n. Show that L (n) ≤ 2n:
Induction Basis: Clearly L (0) = 1 ≤ 20 since a tree of height 0 has at most one leaf. Induction Hypothesis: Assume L (k) ≤ 2k for all k, 1 ≤ k ≤ n. Induction Step: Let T be a tree of height n+1. By induction hypothesis, the number of leaves in the left and right subtrees of the root is ≤ 2n. It follows that the number of leaves in T is ≤ 2n + 2n = 2{n+1}.
Lecture # 1 Slide # 21<BUTTONS AREA: Do not write anything here>
Example on Proof by mathematical induction Using the Principle of Mathematical Induction, Prove that a binary tree of height n has at most 2n leaves: Proof: Let L (n) represent the number of leaves of a binary tree of height n. We want to show that L (n) ≤ 2n using mathematical induction:
Induction Basis: Clearly L (0) = 1 ≤ 20 since a tree of height 0 can have no nodes other than the root. That is, it has at most one leaf. Induction Hypothesis: Assume L (k) ≤ 2k for all k, 1 ≤ k ≤ n. Induction Step: Let T be a tree of height n+1. By induction hypothesis, the number of leaves in the left and right subtrees of the root is ≤ 2n. It follows that the number of leaves in T is ≤ 2n + 2n = 2{n+1}. Thus, the property is true for (n +1). Since T is arbitrary, the proof is complete.
ICS-353 Lecture Template
Logarithms
• Importance of the logarithm function.
• Definition of the logarithm function
• The base is omitted when it is = 2.
• The natural number e, defined by
... 7182818.2...!3
1!2
1!1
1111lim =++++=
+=
∞→
n
n ne
• loge x is written as ln x , and is called the natural logarithm of x. • ln x is also defined by:
. 11
dtt
xx
∫=ln
Lecture # 1 Slide # 22<BUTTONS AREA: Do not write anything here>
Logarithms • The logarithm function, and specifically to the base 2, is used very extensively in
the analysis of many algorithms.
• Definition. Let b > 1, be a positive real number, x a real number, and suppose that for some positive real number y we have: y = bx. Then, x is called the logarithm of y to the base b and is written as x = log . yb
• Since the log to the base 2 is used most often in computational complexity,
therefore, we omit the base when it is = 2. • Another useful base is the natural number e, which is defined by
... 7182818.2...!3
1!2
1!1
1111lim =++++=
+=
∞→
n
n ne
• Generally, loge x is written as ln x , and is called the natural logarithm of x. • ln x is also defined by:
. 1ln1
dtt
xx
∫=
ICS-353 Lecture Template
Logarithms (Cont.)
Properties of logarithms • xy = log b x + log b y, and log b (cblog y) = y log b c, if c > 0 . • is a strictly increasing function. blog
• is a one-to-one function. blog
• . 01log =b
• log e ≈ 1.443 and log 10 ≈ 3.32.
• The derivative of ln (x) is 1/x.
• The derivative of log (x) = log (e) / x.
• . ab a
b =log
Lecture # 1 Slide # 23<BUTTONS AREA: Do not write anything here>
Properties of logarithms
For any real numbers x and y greater than 0, the following properties of logarithms follow from the definition:
• log b xy = log b x + log b y, and log b (cy) = y log b c, if c > 0 . • is a strictly increasing function, that is, if x > y, then log . blog yx bb log>• is a one-to-one function, that is, if log , then x = y. blog y log b=xb
• . 01log =b
• log e ≈ 1.443. • log 10 ≈ 3.32. • The derivative of ln (x) is 1/x. • The derivative of log (x) = log (e) / x. • . ab a
b =log
ICS-353 Lecture Template
Logarithms (Cont.)
Properties of logarithms (Cont.)
• Converting between bases:
bx
xa
ab log
loglog = .
For example:
2lnlnlog xx = and
exx
loglogln = .
• Important identity:
xy bb yx loglog = , x, y > 0.
Lecture # 1 Slide # 24<BUTTONS AREA: Do not write anything here>
More properties of logarithms
• To convert from one base to another, we use the following chain rule:
bx
xa
ab log
loglog = .
For example:
2lnlnlog xx = and
exx
loglogln = .
• The following identity can be proved by taking the logarithms of both sides:
xy bb yx loglog = , x, y > 0.
ICS-353 Lecture Template
Floor and Ceiling Functions
• Definitions:
- Monotonically increasing function - Monotonically decreasing function.
- Strictly increasing function.
- Strictly decreasing function.
• The floor and ceiling functions to be defined later are monotonically increasing.
Lecture # 1 Slide # 25<BUTTONS AREA: Do not write anything here>
Floor and Ceiling Functions
• Definitions:
- A function is monotonically increasing if m ≤ n implies f (m) ≤ f (n). - A function is monotonically decreasing if m ≤ n implies f (m) ≥ f (n).
- A function is strictly increasing if m < n implies f (m) < f (n).
- A function is strictly decreasing if m < n implies f (m) > f (n).
• The floor and ceiling functions to be defined later are monotonically increasing.
ICS-353 Lecture Template
Floor and Ceiling Functions (Cont.)
• Let x be any real number, then: - is defined as the greatest integer ≤ x. x- is defined as the smallest integer ≥ x. xExamples:
27.2 =
37.2 =
35.2 −=−
25.2 −=−
• .2/2/ xxx =+• . xx −=−
• . xx −=−• x-1 < ≤ x ≤ < x + 1. x mn /
Lecture # 1 Slide # 26<BUTTONS AREA: Do not write anything here>
• Let x be any real number, then:
- The floor of x, denoted by is defined as the greatest integer ≤ x. x- The ceiling of x, denoted by is defined as the smallest integer ≥ x. xExamples:
27.2 = 37.2 =
35.2 −=− 25.2 −=−
Few more important properties are: • .2/2/ xxx =+• . xx −=−
• . xx −=−• x-1 < ≤ x ≤ < x + 1. x mn /
ICS-353 Lecture Template
Floor and Ceiling Functions (Cont.)
• Theorem. Let f (x) be a monotonically increasing function such that if f (x) is
integer, then x is integer. Then, )()( xfxf =
and . )()( xfxf = Examples: for any positive integer n and integers a ≠ 0 and b ≠ 0:
- = . nx / nx /- = . nx / nx /- . abnban /// =- . abnban /// =- . 8/2/4/2/2/2/ nnn ==
• If n is not a power of 2, then there is an integer k such that 2k < n < 2 k + 1. In this case, and . nk log= 1log += kn
• The following two inequalities hold: - . nn n 22 log <≤
- nn n ≤< log22
Lecture # 1 Slide # 27<BUTTONS AREA: Do not write anything here>
• Theorem. Let f (x) be a monotonically increasing function such that if f (x) is integer, then x is integer. Then,
)()( xfxf =
and . )()( xfxf = Examples: for any positive integer n and integers a ≠ 0 and b ≠ 0:
- = . nx / nx /- = . nx / nx /- . abnban /// =- . abnban /// =- . 8/2/4/2/2/2/ nnn ==
• If n is not a power of 2, then there is an integer k such that 2k < n < 2 k + 1. In this case, and . nk log= 1log += kn
• The following two inequalities hold: - . nn n 22 log <≤
- nn n ≤< log22
ICS-353 Lecture Template
Factorial and Binomial coefficients
• General
• Factorials - Permutation of a set of n distinct objects - k-permutation
- When k = n, the quantity becomes
n
nP = 1 * 2 *… * n,
and is called “n factorial” and denoted by n!.
- Useful approximation to n! is Stirling’s formula nenn )/(*2( π . - Using Stirling’s formula, we compute 30! with relative error of about 0.27%
only.
Lecture # 1 Slide # 28<BUTTONS AREA: Do not write anything here>
• Factorial and Binomial coefficients We briefly list some of the important combinatorial properties that are frequently
used in the analysis of algorithm. These properties are used with combinatorial problems.
n
kP
In particular, we will limit our review to permutations and combinations, which lead to the definitions of factorials and binomial coefficients.
• Factorials - A permutation of a set of n distinct objects is defined as an ordered
arrangement of these objects. For example, in the 3-elements set {a, b, c} there are 6 permutations: abc, acb, bac, bca, cab, cba.
- We may be also interested in ordered arrangements of some of the elements
of a set. In this case, we use the term k-permutation to refer to an ordered arrangement of k elements of a set. The number of ways to choose k ≤ n objects in an ordered arrangement (also, called the number of permutations of n objects taken k at a time) is:
n (n-1) (n-2) …(n – k +1) and denoted by or P (n, k)
ICS-353 Lecture Template
This is not a new slide
Please note that the spoken part below belongs to the previous slide. There was no space over there.
Lecture # 1 Slide # 29<BUTTONS AREA: Do not write anything here>
- When k = n, the quantity becomes n
nP = 1 * 2 *… * n,
and is called the number of permutations of n objects. Because of its importance, this quantity is designated as “n factorial” and denoted by n!. By convention, 0! = 1, which gives the following simple recursive definition of n!:
- 0! = 1, n! = n (n – 1) ! if n ≥ 1. - A useful approximation to n! is Stirling’s formula nenn )/(*2( π . - For example, using Stirling’s formula, we compute 30! with relative error of
about 0.27% only.
ICS-353 Lecture Template
Binomial coefficients
• Combinations (also, referred to as “ k-combinations “ or “ n choose k”) denoted by C . n
k
Example: the combinations of the four letters a, b, c and d taken three at a time
• This is in fact the unordered selection of 3 elements from the 4-elements set.
• 0, )!(!
!!
≥≥−
== knknk
nkP
Cn
knk
• Other common notations for C are C (n, k) and nk
kn
• Binomial coefficient. • This number is equivalent to the number of k-element subsets in a set of n elements.
Example, Subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}.
Lecture # 1 Slide # 30<BUTTONS AREA: Do not write anything here>
Binomial coefficients • Another important concept is the combination of n objects taken k at a time (also,
referred to as “ k-combinations “ or “ n choose k”). It refers to the possible ways to choose k objects out of n objects, disregarding order, and denoted by C . n
k
Example: the combinations of the four letters a, b, c and d taken three at a time are: a b c, a b d, a c d, b c d.
• This is in fact the unordered selection of 3 elements from the 4-elements set. • Since order does not matter here, the combinations of n objects taken k at a time is equal
to the number of permutations of n objects taken k at a time divided by k!. That is
0, )!(!
!!
≥≥−
== knknk
nkP
Cn
knk
• Other common notations for are C (n, k) and nkC
kn
• This number is also called the Binomial coefficient. This name is used because these numbers occur as coefficients in the expansion of powers of binomial expressions such as (x+ y)n.
• This number is equivalent to the number of k-element subsets in a set of n elements. For example, the 3-element subsets of the set {a, b, c, d} are : {a, b, c}, {a, b, d}, {a, c, d}, {b, c, d}.
ICS-353 Lecture Template
Binomial coefficients (Cont.)
• Important properties of Binomial coefficients:
- The number of ways to choose k elements out of n elements is equal to the number of ways not to choose n-k elements out of n elements. This implies that:
k
n
− kn
n
n
n
0
n
k
n
−
k
n 1
−
−
1
1
k
n
= , in particular
=
= 1
- The following identity is also important one:
= +
Lecture # 1 Slide # 31<BUTTONS AREA: Do not write anything here>
• Some of the important properties of binomial coefficients:
- The # of ways to choose k elements out of n elements is equal to the number of ways not to choose n-k elements out of n elements. This implies that:
k
n
− kn
n
n
n
0
n
k
n
−
k
n 1
−
−
1
1
k
n
= , in particular
=
= 1
This means that the n-combination (= the number of n objects taken n at a time) equals the 0-combination (= the number of combinations of n objects
taken 0 at a time) and equals to 1.
- The following identity is also important one:
= +
That is, the number of combinations of n objects taken k at a time equals to the number of combinations of n - 1 objects taken k at a time + the number of combinations of n - 1 objects taken k - 1 at a time.
ICS-353 Lecture Template
Binomial coefficients (Cont.)
• Properties of Binomial coefficients (Cont.)
- Pascal’s Identity. Used in the geometric arrangement of the binomial coefficients in Pascal triangle.
- It is stated as follows. Let n and k be positive integers with n ≥ k. Then
+
k
n 1 =
+
−1k
n
k
n
The nth row in Pascal’s triangle consists of the binomial coefficients , k = 0,
1, ..., n as follows:
kn
Lecture # 1 Slide # 32<BUTTONS AREA: Do not write anything here>
• Properties of binomial coefficients (Cont.)
- Pascal’s Identity. It is used in the geometric arrangement of the binomial coefficients in a triangle called Pascal triangle shown later.
- Pascal’s Identity is stated as follows. Let n and k be positive integers with n
≥ k. Then
+
k
n 1 =
+
−1k
n
k
n
The nth row in Pascal’s triangle consists of the binomial coefficients
, k = 0, 1, ..., n as shown in the next slide.
kn
ICS-353 Lecture Template
Binomial coefficients (Cont.)
• Properties of Binomial coefficients (Cont.)
1
00
1 1
11 0
1
1 2 1
22 1
2 02
1 3 3 1
33 2
3 13 0
3
1 4 6 4 1
44 3
4 24 1
4 04
The first 5 rows in Pascal’s Triangle.
Lecture # 1 Slide # 33<BUTTONS AREA: Do not write anything here>
1
00
1 1
11 0
1
1 2 1
22 1
2 02
1 3 3 1
33 2
3 13 0
3
1 4 6 4 1
44 3
4 24 1
4 04
The first 5 rows in Pascal’s Triangle.
ICS-353 Lecture Template
Binomial coefficients (Cont.)
• Properties of Binomial coefficients (Cont.)
- Binomial theorem. It gives coefficients of the expansion of powers of binomial expressions.
- Binomial theorem statement: Let n be a positive integer, and x, y be two variables then:
. jjnn
j
n yxjnyx −
=∑
=+ )(
0
= .nnnnn ynnxyn
nyxnyxnxn
+
−++
+
+
−−− 1221
1 ...210
Lecture # 1 Slide # 34<BUTTONS AREA: Do not write anything here>
• Properties of binomial coefficients (Cont.)
- A fundamental tool in the analysis of algorithms is the binomial theorem. It gives coefficients of the expansion of powers of binomial expressions, where a binomial expression is simply the sum of two terms, such as x + y.
- The binomial theorem is stated as follows: Let n be a positive integer, and x,y be two variables then:
. jjnn
j
n yxjnyx −
=∑
=+ )(
0
= .nnnnn ynnxyn
nyxnyxnxn
+
−++
+
+
−−− 1221
1 ...210
ICS-353 Lecture Template
Binomial coefficients (Cont.)
Examples on Binomial theorem: Example 1: Find the expansion of (x + y)3. Solution: Using the binomial theorem, we get:
(x + y)3 = . jj
jyxj
−
=∑
3
3
0 3
= C (3, 0) x3 + C(3, 1) x2y + C (3, 2) xy2 + C (3, 3) y3 = x3 + 3x2y + 3xy2 + y3
Example 2: What is the coefficient of xy3 in the expansion (x + y)4? Solution: Using the binomial theorem, we have the coefficient as follows:
34 = 4
!1 !3!4
)!34(!3!4
==−
Lecture # 10 Slide # 35<BUTTONS AREA: Do not write anything here>
Examples on Binomial theorem: Example 1: Find the expansion of (x + y)3. Solution: Using the binomial theorem, we get:
(x + y)3 = . jj
jyxj
−
=∑
3
3
0 3
= C (3, 0) x3 + C(3, 1) x2y + C (3, 2) xy2 + C (3, 3) y3 = x3 + 3x2y + 3xy2 + y3
Example 2: What is the coefficient of xy3 in the expansion (x + y)4? Solution: Using the binomial theorem, we have the coefficient as follows:
34 = 4
!1 !3!4
)!34(!3!4
==−
ICS-353 Lecture Template
Binomial coefficients (Cont.)
If x = 1, then we get:
. )1(0
jn
j
n yjny ∑
=
=+
Making x =1 and y = 1, then we get, j
n
j
njn )1( )11(
0∑
=
=+
that is nnnnn 2...10 =
++
+
This states that the number of all subsets of a set of size n is equal to 2n.
If we let x = -1 and y = 1, then we get:
0)1()1()1(- )1)1((00
=−
=
=+− ∑∑
=
−
=
jn
j
jnjn
j
njn
jn
That is or . 0...110 =
±−
+
−
nnnnn ∑∑
=
n
oddj
n
evenjjn
jn
Lecture # 1 Slide # 36<BUTTONS AREA: Do not write anything here>
If we let x = 1 in the Binomial theorem above, then we get the following important identity:
. )1(0
jn
j
n yjny ∑
=
=+
Making x =1 and y = 1, in the above theorem, then we get, j
n
j
njn )1( )11(
0∑
=
=+
that is nnnnn 2...10 =
++
+
In terms of combinations, this identity states that the number of all subsets of a set of size n is equal to 2n, as expected.
If we let x = -1 and y = 1 in the binomial theorem, then we get:
0)1()1()1(- )1)1((00
=−
=
=+− ∑∑
=
−
=
jn
j
jnjn
j
njn
jn
That is , or . 0...110 =
±−
+
−
nnnnn ∑∑
=
n
oddj
n
evenjjn
jn
ICS-353 Lecture Template
Binomial coefficients (Cont.)
• From the Binomial theorem above, we have:
(1 + x) 2 = 1 + 2x + x2; (1+x)3 = 1 + 3x + 3x2 + x3 and so on. • Continuing this way indefinitely, we get the elements of the Pascal triangle, where:
Each row after the first is computed from the previous row using the equation
= +
k
n
−
k
n 1
−
−
1
1
k
n
• The Binomial coefficients are the digits of the number (1 + b)n written in base b
notation, if b is sufficiently large (e.g., 10).
Example: Consider base 10, where 11 = (1 + 10), we have: (11)0 = 1, (11)1 = 11, (11)2 = 1+2*10+(10)2 = 121, (11)3 = 1331, (11)4 = 14641 and so on.
Lecture # 1 Slide # 37<BUTTONS AREA: Do not write anything here>
• From the Binomial theorem above, we have: (1 + x) 2 = 1 + 2x + x2; (1+x)3 = 1 + 3x + 3x2 + x3 and so on.
• Continuing this way indefinitely, we get the elements of the Pascal triangle
shown previously, where:
Each row after the first is computed from the previous row using the equation
= +
k
n
−
k
n 1
−
−
1
1
k
n
• It is observed that the binomial coefficients are the digits of the number (1 + b)n
written in base b notation, if b is sufficiently large (e.g., 10).
Example: Consider base 10, where 11 = (1 + 10), we have: (11)0 = 1, (11)1 = 11, (11)2 = 1+2*10+(10)2 = 121, (11)3 = 1331, (11)4 = 14641 and so on., where the digits 1,4,6,4,1 in the last number for example are the entries in the fifth row of Pascal’s triangle!!!
ICS-353 Lecture Template
The Pigeonhole Principle
• An easy, intuitive and extremely powerful. • States that: If k + 1 or more objects are placed into k boxes, then there is at least one
box containing two or more of the objects.
• Theorem: If n objects are distributed into m boxes, then: - One box contain at least objects and one box must contain at most
objects. mn /
mn /Example: Suppose that G = (V, E) be a connected undirected graph with m vertices. Also, let p be a path in G that visits n > m vertices. Using the Pigeonhole Principle, we show that p must contain a cycle.
Solution: Since the ≥ 2, there is at least one vertex, say v that is visited by p mn / more than once. Thus, the portion of the path that starts and ends at v constitutes a cycle.
Lecture # 1 Slide # 38<BUTTONS AREA: Do not write anything here>
The Pigeonhole Principle (also called: the Dirichlet drawer principle) • It is an easy, intuitive and extremely powerful principle and is essential in the
analysis of algorithms. In the 19th century, the German mathematician Dirichlet used this principle in his work.
• The Pigeonhole Principle states that: If k + 1 or more objects are placed into k boxes, then there is at least one box containing two or more of the objects.
• Theorem: If n objects are distributed into m boxes, then:
- One box contain at least objects and one box must contain at most objects.
mn /
mn /Example on the Pigeonhole Principle: Suppose that G = (V, E) be a connected undirected graph with m vertices. Also, let p be a path in G that visits n > m vertices. Using the Pigeonhole Principle, we show that p must contain a cycle.
Solution: Since the ≥ 2, there is at least one vertex, say v that is visited mn /
by p more than once. Thus, the portion of the path that starts and ends at v constitutes a cycle.
ICS-353 Lecture Template