8/4/2019 cs161-02

1/10

1

19

Analyzing Insertion Sort as aAnalyzing Insertion Sort as aRecursive AlgorithmRecursive Algorithm

l Basic idea: divide and conquer Divide into 2 (or more) subproblems. Solve each subproblem recursively. Combine the results.

l Insertion sort is just a bad divide & conquer ! Subproblems: (a) last element

(b) all the rest Combine: find where to put the last element

Lecture 2, April 5, 2001

20

Recursion for Insertion SortRecursion for Insertion Sort

l We get a recursion for the running time T(n):

l Formal proof: by induction.

l Another way of looking: split into n subproblems, mergeone by one.

12

( 1) for 1( )1 for 1

( ) ( 1)( 2) ( 1)( 3) ( 2) ( 1)

...

( )

n

i

T n n nT nn

T n T n nT n n nT n n n n

i

n=

+ >= == += + += + + +===

8/4/2019 cs161-02

2/10

2

21

Improving the insertion sortImproving the insertion sort

l Simple insertion sort is good only for small n.

l Balance sorting vs. merging: Merge equal size chunks.

l How to merge:i=1, j=1for k=1 to 2n

if A(i)

8/4/2019 cs161-02

3/10

8/4/2019 cs161-02

4/10

4

25

More summationsMore summations

l Another useful trick:

l Summary: Learn to recognize standard simplifications Try going opposite direction If all fails - apply tricks one by one. . .

kx xd dx

x xd dx x

x

xk

k

k

k

= = = =

=

0 0

2

11 1( )

26

RecurrenciesRecurrencies

l Chapter 4 in the textbook.

l Algorithm calls itself - recursive .

l First, solve for Claim: Proof by induction:

1 1

( )1

2

n

T nnT otherwise

==

+

n k =2T n n( ) lg= +1

T T T

k

k k

k

k

( )( ) ( )

lg( )

lg( )

1 12 2 1

2 1 12

2 1

1

1

== +

= + += += +

+

+ QED

8/4/2019 cs161-02

5/10

5

27

What if n not a power of 2 ?What if n not a power of 2 ?

l Easy to prove by induction that

l Now we can say:

l Observe that we did not prove Theta , only big- Oh !

l Technically, we should be careful about floor/ceiling, butusually we can safely concentrate on n=power of 2 .

T n T n( ) ( ) 1

lg( ) (2 ) lg 1 (log )

nT n T n n

= + = Q

28

Guessing the solutionGuessing the solution

l Instead of adding sequentially, lets divide into 2 parts,add each one recursively, and add the result:

l Need a stronger induction hypothesis !Assume:Then:

( ) / 2 / 2 1T n T n T n = + +Note that we omit the n=1case for simplicity

Guess: ( ) fo r some constant

Then: ( ) / 2 /2 1

/2 / 2 1

1 ....

T n cn c

T n T n T n

c n c n

cn Oopssss

L 2 1 1for

8/4/2019 cs161-02

6/10

6

29

Another exampleAnother example

l Consider recursion:

l First guess:

l We omit base case.Induction step:

l But we can do better !

( ) 4 2nT n T n

= +

T n cn( ) 3

33 34 ( )2 2 2

for 2, 1 " " 0 QEDrest

n n cT n c n cn n n

c n rest

+ + = + -

1442 443

2

21 2

22

1 2 1 2

21 2 2

First try: ( ) is too weak !

Assume: ( )

Then: ( ) 4 4 22 2 2

( ) REST

T n cn

T n c n c nn n nT n T n c c n c n c n n

c n c n n c n

-= + - + = - +

= - + -14243

30

Initial ConditionsInitial Conditions

l Can initial conditions affect the solution ? YES !

l n was assumed to be a power of 2 .

2( ) ( / 2)

(1) 2 ( ) 2(1) 3 ( ) 3(1) 1 ( ) 1

n

n

T n T n

T T nT T nT T n

=

= == == =

8/4/2019 cs161-02

7/10

7

31

IteratingIterating recurrenciesrecurrencies

l Example:

l Disadvantages: Tedious

Error- pronel Use to generate initial guess , and then prove by

induction !

T n T n nn n T n n n T nn n n T n n n n T n

n n n n n T k i

nn

nn

( ) ( / )( / ( / )) ( / )

[ / ( / )] ( / )

( )lg

lg

lg

= += + + = + += + + + = + + +

= + + + + = +=

-

--

4 24 2 4 4 2 16 42 16 4 4 8 2 4 4 8

2 4 8 2 4 10

1

2 12 1

L

1 24 3 4

1 24 34

Q( )n 2 Q( )n 2

32

Recursion TreeRecursion Treel Example:

l At k - th level we get a general formula: i steps right, k- i left

l Summing over all k, geometric sum, sums to(overcount, since T(1)=1)

T n T n T n n( ) ( / ) ( / )= + +4 2 2

n2

4nT 2

nT

n2

2

4n

2

2n

16nT 8

nT 8nT 4

nT

n2

516

2n

25256

2n

2( ) ( )2 2

2 2

2 4 4 16

1 1 54 16 16

k i k ii i

i ik k

k k n ni i

n n

- - - -- -

= =

= + =

Q( )n2

8/4/2019 cs161-02

8/10

8

33

Master MethodMaster Method

l Consider the following recurrence:

l More general than the book.

l

Let . Then the cases are: Q polynomially larger than f . f is larger than Q by a polylog factor. Q polynomially smaller than f .

lg lg

lg lg 1

lg

1. ( ) ( ), 0 ( )

2. ( ) ( lg ), 0 ( lg )

( ) ( ), 03. ( ( ))( / ) ( ) fo r some 1

b b

b b

b

a a

a a k k

a

f n O n n

f n n n k n n

f n n f n af n b cf n c

+

+

= >

=

= >

Q n b a= lg

34

Build recursion treeBuild recursion tree

f n( )

f n b( / ) n b( / )

f n b( / )2 n b( / )2 n b( / )2 f n b( / )2... ...

n( )

a f n b( / )

a n b2 2( / )

...

Last row: ( elements, each one

Total:

a n

n a f n b

b b

bb

n a

a i i

i

n

lg lg

lglg

) ( ) ( ).

( ) ( / )

=

+=

1

1

1

Which term dominates ?

8/4/2019 cs161-02

9/10

9

35

First case: f(n) smallFirst case: f(n) small

n f n

n c f n cn n

a f n b ca n b cn a bb

cn b

bb

n

O n

n

bb

b a b

b

b

b

b

b

aa

j j j j a j j

j aa j

n

a

a

lglg

lglg

lg

lg

lg

lg

( )( ) ( ) /

( / ) ( / )

( ).

( ).

( ).)

lg

= $

= =

-- =

- - -

W

Q

Q

e e

ee

e e

e

e

e

e

s.t for "large enough n",

The ratio summed over all possible j:

Total:

Lower bound is trivial (Why ?? First term in the original expression was already

11

36

Second caseSecond case

lg

lg

lglg1

lg

( ) ( lg )

lg (lg ) (there are (lg ) elements in the sum)

This is an UPPER bound ! How to prove the lower bound ??

Rough and easy ap

b

bb

a k b

a k

aa j k k

j j

nn

f n n n

n na O n n O nb b

+

=

= Q

=1442443 14243

lg 1 (lg )/2 (lg )/21

1 1 1

proach:

lg lg lg (const) lg

(Note that we use the assumption that 0)

b b bn n nk k k k

j ji i i

n n n nb b

k

-+

= = =

=

8/4/2019 cs161-02

10/10

10

37

Third caseThird case

a f n b c f n c f n n

c f n f n

a f n b O f n

n O f n

f n

j j j a

j

i

n

j j

i

n

a

b

b

b

b

( / ) ( ) , ( ) ( )

( ) ( ( ))

( / ) ( ( ))

( ) ( ( ))

( ( ))

lg

lg

lg

lg

< =

=

=

=

+

=

=

for some and

Note Big- Oh and not Theta !

The first term is already

TOTAL:

1

1

1

1

1