data structure and algorithms design

BITS PilaniPilani Campus

Data Structure and Algorithms Design

Dr. Maheswari Karthikeyan Lecture2

BITS Pilani, Pilani Campus

A way to describe behavior of functions in the limit

– How we indicate running times of algorithms

– Describe the running time of an algorithm as n grows to

O notation: asymptotic “less than”: f(n) “≤” g(n)

notation: asymptotic “greater than”: f(n) “≥” g(n)

notation: asymptotic “equality”: f(n) “=” g(n)

Asymptotic Notations


For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine which relationship is correct.– f(n) = log n2; g(n) = log n + 5

– f(n) = n; g(n) = log n2

– f(n) = log log n; g(n) = log n

– f(n) = n; g(n) = log2 n

Asymptotic Notations - Examples

f(n) = (g(n))f(n) = (g(n))f(n) = O(g(n))

f(n) = (g(n))


• Intuitively: O(g(n)) = the set of functions with a smaller or same order of growth as g(n)

Asymptotic notations

O-notation


Examples

3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2

3n + 3 = O(n) ;

100n + 6 = O(n) ;

3n + 3 <= 4n for all n >= 3

100n + 6 <= 101n for all n >= 6

= O(n2)

10 n 2 + 4n + 2 < = 11 n2 for n >= 510 n 2 + 4n + 2


- notation

Asymptotic notations (cont.)

• Intuitively: (g(n)) = the set of

functions with a larger or same order

of growth as g(n)


3n + 2 = ?

3n + 3 = ?

100n + 6 = ?

Examples

3n + 2 >= 3n for all n >= 1

3n + 3 >= 3n for all n >= 1

100n + 6 >= 100n for all n >= 1


-notation

Asymptotic notations (cont.)

• Intuitively (g(n)) = the set of

functions with the same order of

growth as g(n)


Iterative methods:Insertion sortBubble sortSelection sort

Sorting

Divide and conquer • Merge sort• Quicksort

Non-comparison methods• Counting sort• Radix sort• Bucket sort


To insert 12, we need to make room for it by moving first 36 and then 24.

Insertion Sort

6 10 24

12

36


6 10 24

Insertion Sort

36

12


Insertion Sort

6 10 24 36

12


Insertion Sort

5 2 4 6 1 3

input array

left sub-array right sub-array

at each iteration, the array is divided in two sub-arrays:

sorted unsorted


Insertion Sort


Alg.: INSERTION-SORT(A)for j ← 2 to n do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > keydo A[i + 1] ← A[i] i ← i – 1 A[i + 1] ← key

Insertion sort – sorts the elements in place

Insertion Sort

a8a7a6a5a4a3a2a1

1 2 3 4 5 6 7 8

key


n

j jt2

n

j jt2)1(

n

j jt2)1(

)1(11)1()1()( 82

72

62

5421

nctctctcncncncnTn

jj

n

jj

n

jj

tj: # of times the while statement is executed at iteration j

Analysis of Insertion Sort

INSERTION-SORT(A)for j ← 2 to n

do key ← A[ j ] Insert A[ j ] into the sorted sequence A[1 . . j -1] i ← j - 1 while i > 0 and A[i] > key

do A[i + 1] ← A[i] i ← i – 1

A[i + 1] ← key

cost times

c1 n c2 n-

1 0 n-

1 c4 n-

1 c5

c6 c7

c8 n-1


The array is already sorted– A[i] ≤ key upon the first time the while loop test is run (when i = j -1)

– tj = 1

T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1) = (c1 + c2 + c4 + c5 + c8)n + (c2 + c4 + c5 + c8)

= an + b = (n)

Best Case Analysis

“while i > 0 and A[i] > key”

)1(11)1()1()( 82

72

62

5421

nctctctcncncncnTn

jj

n

jj

n

jj


The array is in reverse sorted order– Always A[i] > key in while loop test– Have to compare key with all elements to the left of the j-th

position compare with j-1 elements tj = j

Worst Case Analysis

using1 2 2

( 1) ( 1) ( 1)1 ( 1)2 2 2

n n n

j j j

n n n n n nj j j

we have:

)1(2)1(

2)1(1

2)1()1()1()( 8765421

ncnncnncnncncncncnT

cbnan 2a quadratic function of n

T(n) = (n2) order of growth in n2

)1(11)1()1()( 82

72

62

5421

nctctctcncncncnTn

jj

n

jj

n

jj


Alg.: SELECTION-SORT(A)n ← length[A]for j ← 1 to n - 1

do smallest ← j for i ← j + 1 to n

do if A[i] < A[smallest] then smallest ← i

exchange A[j] ↔ A[smallest]

Selection Sort

1329648


»n2/2 comparisons

Alg.: SELECTION-SORT(A)

n ← length[A] for j ← 1 to n - 1

do smallest ← j for i ← j + 1 to n do if A[i] < A[smallest] then smallest ← i exchange A[j] ↔ A[smallest]

Analysis of Selection Sortcost

times

c1 1 c2 n c3 n-

1 c4

c5

c6 c7 n-

1

1

1)1(n

jjn

1

1)(n

jjn

1

1)(n

jjn»n

exchanges

1 1 1

21 2 3 4 5 6 7

1 1 2

( ) ( 1) ( 1) ( 1) ( )n n n

j j j

T n c c n c n c n j c n j c n j c n n


Divide the problem into a number of sub-problems– Similar sub-problems of smaller size

Conquer the sub-problems– Solve the sub-problems recursively

– Sub-problem size small enough solve the problems in

straightforward manner

Combine the solutions to the sub-problems– Obtain the solution for the original problem

Divide-and-Conquer


To sort an array A[p . . r]:Divide

– Divide the n-element sequence to be sorted into two subsequences of n/2 elements each

Conquer– Sort the subsequences recursively using merge sort– When the size of the sequences is 1 there is nothing more to do

Combine– Merge the two sorted subsequences

Merge Sort Approach


Alg.: MERGE-SORT(A, p, r)if p < r Check for base

case

then q ← (p + r)/2 Divide

MERGE-SORT(A, p, q) Conquer

MERGE-SORT(A, q + 1, r) Conquer

MERGE(A, p, q, r) Combine

Initial call: MERGE-SORT(A, 1, n)

Merge Sort

1 2 3 4 5 6 7 8

62317425

p rq


Example – n Power of 21 2 3 4 5 6 7 8

q = 462317425

1 2 3 4

74255 6 7 8

6231

1 2

253 4

745 6

317 8

62

1

52

23

44

7 16

37

28

65

Example


Example – n Power of 2

1

52

23

44

7 16

37

28

65

1 2 3 4 5 6 7 8

76543221

1 2 3 4

75425 6 7 8

6321

1 2

523 4

745 6

317 8

62


Example – n Not a Power of 2

625374162741 2 3 4 5 6 7 8 9 10 11

q = 6

4162741 2 3 4 5 6

625377 8 9 10 11

q = 9q = 3

2741 2 3

4164 5 6

5377 8 9

6210 11

741 2

23

164 5

46

377 8

59

210

611

41

72

64

15

77

38


Example – n Not a Power of 2

776654432211 2 3 4 5 6 7 8 9 10 11

7644211 2 3 4 5 6

765327 8 9 10 11

7421 2 3

6414 5 6

7537 8 9

6210 11

23

46

59

210

611

41

72

64

15

77

38

741 2

614 5

737 8


Input: Array A and indices p, q, r such that p ≤ q < r– Subarrays A[p . . q] and A[q + 1 . . r] are sorted

Output: One single sorted subarray A[p . . r]

Merging

1 2 3 4 5 6 7 8

63217542

p rq


Idea for merging:– Two piles of sorted cards

• Choose the smaller of the two top cards

• Remove it and place it in the output pile– Repeat the process until one pile is empty– Take the remaining input pile and place it face-down onto the output pile

Merging


MERGE(A, 9, 12, 16)p rq


Example: MERGE(A, 9, 12, 16)


Example (cont.)


Example (cont.)

Done!


Alg.: MERGE(A, p, q, r)1. Compute n1 and n2

2. Copy the first n1 elements into L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1]

3. L[n1 + 1] ← ; R[n2 + 1] ← 4. i ← 1; j ← 15. for k ← p to r6. do if L[ i ] ≤ R[ j ]7. then A[k] ← L[ i ]8. i ←i + 19. else A[k] ← R[ j ]10. j ← j + 1

Merge - Pseudocode

p q

7542

6321rq + 1

L

R

1 2 3 4 5 6 7 8

63217542

p rq

n1 n2


Initialization (copying into temporary arrays):– (n1 + n2) = (n)

Adding the elements to the final array (the last for loop):– n iterations, each taking constant time (n)

Total time for Merge:– (n)

Running Time of Merge


The recurrence is based on the three steps of the paradigm:– T(n) – running time on a problem of size n– Divide the problem into a subproblems, each of size n/b: takes D(n)– Conquer (solve) the subproblems aT(n/b) – Combine the solutions C(n)

(1) if n ≤ c T(n) = aT(n/b) + D(n) + C(n) otherwise

Analyzing Divide-and Conquer Algorithms


Divide: – compute q as the average of p and r: D(n) = (1)

Conquer: – recursively solve 2 subproblems, each of size n/2 2T (n/2)

Combine: – MERGE on an n-element subarray takes (n) time C(n) = (n)

(1) if n =1 T(n) = 2T(n/2) + (n) if n > 1

MERGE-SORT Running Time


T(n) = c if n = 12T(n/2) + cn if n > 1

Use Master’s Theorem:

Compare n with f(n) = cnCase 2: T(n) = Θ(nlgn)

Solve the Recurrence

data structure and algorithms design

Documents

pilani campus bits pilani

gn bits pilani

gnbits pilani

pilani campus15alg

pilani campusfor

pilani campusexamples

pilani campusto

pilani campus3n