Big-O Notation

Definition: Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say that f(x) is O(g(x)) if there are constants C and k such that

whenever x > k.

• This is read as “f(x) is big-O of g(x)” or “gasymptotically dominates f.”

Useful Big-O Estimates Involving Logarithms, Powers, and Exponents

• If d > c > 1, then

nc is O(nd), but nd is not O(nc).

• If b > 1 and c and d are positive, then

(logb n)c is O(nd), but nd is not O((logb n)c).

• If b > 1 and d is positive, then

nd is O(bn), but bn is not O(nd).

• If c > b > 1, then

bn is O(cn), but cn is not O(bn).

2

Big-Omega Notation

Definition: Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say that

if there are constants C and k such thatwhen x > k.

• We say that “f(x) is big-Omega of g(x).”• Big-O gives an upper bound on the growth of a function,

while Big-Omega gives a lower bound. • f(x) is Ω(g(x)) if and only if g(x) is O(f(x)). This follows from the

definitions. Ω is the upper case version of the lower case Greek letter ω.

Big-Theta Notation

• Definition: Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. The function if

and .

• We say that “f is big-Theta of g(x)” and also that “f(x) is of order g(x)” and also that “f(x) and g(x) are of the same order.”

• if and only if there exists constants C1

, C2 and k such that C1g(x) < f(x) < C2 g(x) if x > k. This follows from the definitions of big-O and big-Omega.

Θ is the upper case version of the lower case Greek letter θ.

Understanding the Complexity of Algorithms

5

Comparison of Hexadecimal, Octal, and Binary Representations

Each octal digit corresponds to a block of 3 binary digits.Each hexadecimal digit corresponds to a block of 4 binary digits. So, conversion between binary, octal, and hexadecimal is easy.

Initial 0s are not shown

6

Complexity of Problems• Tractable Problem: There exists a polynomial

time algorithm to solve this problem. These problems are said to belong to the Class P.

• Intractable Problem: There does not exist a polynomial time algorithm to solve this problem

• Unsolvable Problem : No algorithm exists to solve this problem, e.g., halting problem.

• Class NP: Solution can be checked in polynomial time. But no polynomial time algorithm has been found for finding a solution to problems in this class.

• NP Complete Class: If you find a polynomial time algorithm for one member of the class, it can be used to solve all the problems in the class.

7

Congruence RelationDefinition: If a and b are integers and m is a positive integer, then a is congruent to b modulo m if m divides a – b.

– The notation a ≡ b (mod m) says that a is congruent to b modulo m.

– We say that a ≡ b (mod m) is a congruence and that m is its modulus.

– Two integers are congruent mod m if and only if they have the same remainder when divided by m.

– If a is not congruent to b modulo m, we write

a ≢ b (mod m)

gcds as Linear Combinations

Bézout’s Theorem: If a and b are positive integers, then there exist integers s and t such that gcd(a,b) = sa + tb.

s and t such that (a/g)s + (b/g)t = 1

Definition: If a and b are positive integers, then integers s and t such that gcd(a,b) = sa + tb are called Bézoutcoefficients of a and b. The equation

gcd(a,b) = sa + tb is called Bézout’s identity.

Étienne Bézout(1730-1783)

9

Caesar Cipher

• The Caesar cipher is one of a family of ciphers called shift ciphers. Letters can be shifted by an integer k, with 3 being just one possibility. The encryption function is

f(p) = (p + k) mod 26

and the decryption function is

f−1(p) = (p−k) mod 26

The integer k is called a key.

10

Block Ciphers• Ciphers that replace each letter of the alphabet by

another letter are called character or monoalphabeticciphers.

• They are vulnerable to cryptanalysis based on letter frequency. Block ciphers avoid this problem, by replacing blocks of letters with other blocks of letters.

• A simple type of block cipher is called the transposition cipher. The key is a permutation σ of the set {1,2,…,m}, where m is an integer, that is a one-to-one function from {1,2,…,m} to itself.

• To encrypt a message, split the letters into blocks of size m, adding additional letters to fill out the final block. We encrypt p1,p2,…,pm as c1,c2,…,cm = pσ(1),pσ(2),…,pσ(m).

• To decrypt the c1,c2,…,cm transpose the letters using the inverse permutation σ−1.

11

Principle of Mathematical Induction

Principle of Mathematical Induction: To prove that P(n) is true for all positive integers n, we complete these steps:–Basis Step: Show that P(1) is true.– Induction Hypothesis (I.H.): State P(k)– Inductive Step: Show that P(k) → P(k + 1)

is true for all positive integers k.To complete the inductive step, assuming

the inductive hypothesis that P(k) holds for an arbitrary integer k, show that must P(k + 1) be true.

12

Strong Induction• Strong Induction: To prove that P(n) is true

for all positive integers n, where P(n) is a propositional function, complete three steps:

– Basis Step: Verify that the proposition P(1) is true.

– Induction Hypothesis (I.H.): State P(k)

– Inductive Step: Show the conditional statement [P(1) ∧ P(2) ∧∙∙∙ ∧ P(k)] → P(k + 1) holds for all positive integers k.

Strong Induction is sometimes called the

second principle of mathematical induction

or complete induction. 13

Structural InductionWe used mathematical induction to prove a result about a recursively defined set. Next we study a more direct form induction for proving results about recursively defined sets.

Definition: To prove a property of the elements of a recursively defined set, we use structural induction. BASIS STEP: Show that the result holds for all elements

specified in the basis step of the recursive definition.

RECURSIVE STEP: Show that if the statement is true for each of the elements used to construct new elements in the recursive step of the definition, the result holds for these new elements.

• The validity of structural induction can be shown to follow from the principle of mathematical induction.

14

Correspondence Principle

If two finite sets can be placed

into 1-1 onto correspondence,

then they have the same size

It’s one of the

most important

mathematical

ideas of all time!

15

If a finite set A

has a k-to-1

correspondence

to finite set B,

then |B| = |A|/k

16

Product Rule (Rephrased)Suppose every object of a set S can be

constructed by a sequence of choices with P1

possibilities for the first choice, P2 for the

second, and so on.

IF 1. Each sequence of choices

constructs an object of type S

2. No two different sequences create the

same object

There are P1P2P3…Pn objects of type S

AND

THEN

17

Number of ways of ordering,

permuting, or arranging r out of n

objects

n choices for first place, n-1 choices for

second place, . . .

n × (n-1) × (n-2) ×…× (n-(r-1))

n!

(n-r)!=

18

n “choose” r

A combination or choice of r out

of n objects is an (unordered) set

of r of the n objects

The number of r combinations of n objects:

n!

r!(n-r)!=

n

r

19

Arrange n symbols: r1 of type 1,

r2 of type 2, …, rk of type k

n

r1

n-r1

r2

…n - r1 - r2 - … - rk-1

rk

n!

(n-r1)!r1!

(n-r1)!

(n-r1-r2)!r2!= …

=n!

r1!r2! … rk!

20

Counting Multisets

=n + k - 1

n - 1

n + k - 1

k

The number of ways

to choose a multiset of

size k from n types of elements is:

21

binomial

expression

Binomial Coefficients

The Binomial Formula

n

1(1+X)n =

n

0X0 + X1 +…+

n

nXn

22

n

1(X+Y)n =

n

0XnY0 + Xn-1Y1

+…+n

nX0Yn

The Binomial Formula

+…+n

kXn-kYk

23

What is the coefficient

of (X1r1X2

r2…Xkrk)

in the expansion of

(X1+X2+X3+…+Xk)n?

n!

r1!r2!...rk!

24