lecture 6 jianjun hu department of computer science and engineering university of south carolina...
TRANSCRIPT
Lecture 6
Jianjun Hu
Department of Computer Science and Engineering
University of South Carolina
2009.9.
CSCE350 Algorithms and Data Structure
Outline
Review of recursive algorithm running time—cheat sheet
Homogeneous Second-Order Linear Recurrence
Brute Force Strategy for Algorithm Design The art of lazy algorithm is to count/estimate its running time Is it doable within given timeframe?
Time Efficiency of Recursive Algorithms
Steps in mathematical analysis of recursive algorithms:
Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best case for input of size n
Set up a recurrence relation and initial condition(s) for C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls).
Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)
Three Recurrence Types We know How to Find the Closed-Form Solution
Please related them to the following algorithms we learned in the last class
• Recursive algorithm for n!• Recursive algorithm for Tower of Hanoi• Recursive algorithm for finding the number of digits in the
binary representation of a decimal integer• Recursive algorithm for finding the Fibbonacci numbers
)2()1()(
)1,1()/()(
)1()(
nTbnTanT
banbnTanT
nnTanTk
k
Theorem Master
Important Recurrence Types:
One (constant) operation reduces problem size by one.T(n) = T(n-1) + c T(1) = dSolution: T(n) = (n-1)c + d linear
A pass through input reduces problem size by one.T(n) = T(n-1) + cn T(1) = dSolution: T(n) = [n(n+1)/2 – 1] c + d quadratic
One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = dSolution: T(n) = c lg n + d logarithmic
A pass through input reduces problem size by half.T(n) = 2T(n/2) + cn T(1) = dSolution: T(n) = cn lg n + d n n log n
A General Divide-and-Conquer Recurrence: Master Theorem
T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(nk)
a < bk T(n) ∈ Θ(nk)
a = bk T(n) ∈ Θ(nk lg n )
a > bk T(n) ∈ Θ(nlog b a)
Note: the same results hold with O instead of Θ.
Solutions to a Homogeneous Second-Order Linear Recurrence with Constant Coefficients
0,0)2()1()( ancTnbTnaT
Characteristic equation
roots
Then
02 cbrar
21, rr
uvvu
nnnTivur
rnrnTrrr
rrnTrr
n
nn
nn
/arctan
)sincos()(
)(
)(
22
2,1
21
2121
and where
complex are if
real is if
real are both and if
Fibonacci numbers
The Fibonacci sequence:0, 1, 1, 2, 3, 5, 8, 13, 21, …
Fibonacci recurrence:F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1
Another example:A(n) = 3A(n-1) – 2A(n-2) A(0) = 1 A(1) = 3
2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation
with constant coefficientswith constant coefficients
How to Find the Constants and
Using the initial conditions
For example, for Fibonacci numbers
F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1
Characteristic equation roots
We have
Using F(0)=0 and F(1)=1
012 rr2
512,1
r
nn
nF
2
51
2
51)(
5/15/1 and
Computing Fibonacci numbers
• Definition based recursive algorithm
• Nonrecursive brute-force algorithm
• Explicit formula algorithm
• Logarithmic algorithm based on formula:
FF((nn-1)-1) F F((nn))
FF((nn)) F F((nn+1)+1)
0 10 1
1 11 1=
n
for n≥1, assuming an efficient way of computing matrix powers.
Another Example: calculating a^n
Construct an algorithm for computing an, where a>0 is a constant and the integer n>=0 is the input size. Then analyze its time efficiency
Basic operation a multiplication of two float numbers
Two choices• Nonrecursive algorithm• Recursive algorithm
Any idea to make it faster, e.g., with a better time efficiency?
Can we solve this problem in time?)(logn
Empirical Analysis of Algorithm Time Efficiency
Why? Mathematical analysis is difficult for many algorithms Difficult for average case analysis Difficult for special type of data distribution
How? Purpose: compare efficiency Efficiency measures M: operation counts or time unit Determine characteristics of typical sample input Implement the algorithm and run it Different sample sizes, multiple samples for each size
Profiling tools: measuring time spent on different segments of program can pinpoint bottleneck in a program..
Empirical Analysis of Algorithm Time Efficiency
Data analysis for the empirical data
Scatter Plot
Now We Get to Chapter 3 -- Brute Force
From now on, we are going to learn some basic and general strategies in designing algorithms to solve some typical computing problems.
We will analyze the efficiency of these algorithms using the tools learned in the past several classes
We will learn how to design algorithms with better efficiency
First, let’s talk about the Brute Force strategies– the simplest
Brute Force Strategy for Algorithm Design
• Brute Force is a straightforward approach to solving a problem, usually directly based on the problem’s statement and definitions of the concepts involved
• In many cases, Brute Force does not provide you a very efficient solution
• Brute Force may be enough for moderate size problems with current computers….
Sorting Algorithm
We have discussed two sorting algorithms: Selection Sort and Insertion Sort
What are the basic idea behind the Selection-Sort algorithm?• Scanning the entire given list to find its smallest element and
swap it with the first element• This is a straightforward solution – Brute Force strategy• What is its time efficiency –
An example: sorting the numbers [89 45 68 90 29 34 17]
See pseudocode in Section 3.1
)( 2n
Selection sort
)()()()( 2nnCnCnC averagebestworst
Insertion Sort
?)(),()(),()( 2 nCnnCnnC averagebestworst
Another Brute-Force Application: Bubble Sort
Compare adjacent elements and exchange them if they are out of order
This the result after the first pass, which moves the largest as the rightmost element
90
?
?
? ?
?
?
173429896845
17903429896845
17349029896845
17342990896845
17342990688945
17342990684589
Algorithm in Pseudocode
What is the time efficiency?
]1[][][]1[
20
20
]1..0[
]1..0[
]1..0[
])1..0[(
jAjAjAjA
in-j
n-i
nA
nA
nA
nABubbleSort
and swap if
do to for
do to for
order ascending in sorted Array :Output //
elements orderable of array An :Input //
sort bubble by array sorts algorithm The //
ALGORITHM
)(1 22
0
2
0
nn-
i
-in-
j
Sequential Search – Brute Force
Find whether a search key is present in an array
What is time efficiency of this algorithm?
1
1
][
0
][
)],..0[(
10
-
ini
ii
KiA
i
KnA
KnASearchSequential
..n-AK
return else
return if
do while
ALGORITHM
][ in key Search //
Brute-Force String Matching
Find a pattern in the text: Pattern – ‘NOT’, text – ‘NOBODY_NOTICED_HIM’
Typical Applications – ‘find’ function in the text editor, e.g., MS-Word, Google search
What is the time efficiency of this algorithm?
1
1
][][
0
0
])1..0[],1..0[(
-
imj
jj
jiTjPmj
j
n-m i
mPnTtchBFStringMa
return
return if
and while
do to for
ALGORITHM
Closest-Pair and Convex Hull Problems by Brute Force
Closest-Pair problem• Given n points in a plane, find the closest pair• How to solve this problem and what is the time efficiency of
this algorithm?
Convex-Hull problem• Convex hull is the tightest convex polygon that bounds a set of
n points in a plane• Convex polygon – any two points in this polygon results in the
inclusion of the segment that links these two points also in this polygon
Convex/NonConvex Polygons
Convex Hull
Imagine a rubber band around a set of nails
Nails touched by the band extreme points
P7
P
PP
P
6
35
1
PP
P
82
4
Solve Convex-Hull Problem
Connect any pair of points by a line segment.
Each line segment partitions the plane to the two half planes
If all the n points are on the same side of this line segment
Such a line segment is an edge of the convex-hull polygon
What is the time efficiency of this Brute-Force algorithm?For each possible pair of points, we need to check whether all
the remaining n-2 points are on the same side of the line segment that connects these pair of points.
For Sorting, String Matching, and Convex-Hull problems, we will revisit them by designing more efficient algorithms.
Exhaustive Search
A brute-force approach to combinatorial problem• Generate each and every element of the problem’s domain• Then compare and select the desirable element that satisfies
the set constraints• Involve combinatorial objects such as permutations,
combinations, and subsets of a given set• The time efficiency is usually bad – usually the complexity
grows exponentially with the input size
Three examples• Traveling salesman problem• Knapsack problem• Assignment problem
Traveling Salesman Problem
Find the shortest tour through a given n cities that visits each city exactly once before returning to the starting city
Using graph model: city vertex, road edge, length of the road edge weight.
TSP shortest Hamiltonian Circuit – a cycle that passes through all the vertices of the graph exactly once
Exhaustive search:• List all the possible Hamiltonian circuits (starting from any
vertex)• Ignore the direction • How many candidate circuits do we have? (n-1)!/2• Very high complexity
TSP Example
a
dc
b
5
2
3
1
8 7
a ---> b ---> c --->d ---> a
a ---> b---> d ---> c ---> a
a ---> c ---> b ---> d --->a
a ---> c ---> d ---> b ---> a
a---> d ---> b ---> c ---> a
a ---> d ---> c ---> b ---> a
l = 2 + 8 + 1 + 7 = 18
l = 2 + 3 + 1 + 5 = 11
l = 5 + 8 + 3 + 7 = 23
l = 5 + 1 + 3 + 2 = 11
l = 7 + 3 + 8 + 5 = 23
l = 7 + 1 + 8 + 2 = 18
optimal
optimal
Tour Length