03 brute force
TRANSCRIPT
-
8/4/2019 03 Brute Force
1/33
Chapter 3Chapter 3
Brute ForceBrute Force
Copyright 2007 Pearson Addison-Wesley. All rights reserved.
-
8/4/2019 03 Brute Force
2/33
3-2Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Brute ForceBrute Force
A straightforward approach, usually based directly on theA straightforward approach, usually based directly on theproblems statement and definitions of the concepts involvedproblems statement and definitions of the concepts involved
Examples:Examples:
1.1. ComputingComputing aann ((aa > 0,> 0, nn a nonnegative integer)a nonnegative integer)
2.2. ComputingComputing nn!!
3.3. Multiplying two matricesMultiplying two matrices
4.4. Searching for a key of a given value in a listSearching for a key of a given value in a list
-
8/4/2019 03 Brute Force
3/33
3-3Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Importance of Brute Force?
Applicable to a very wide variety of problems. For some important problems, it yields reasonable
algorithms.
The expense of designing a more efficient
algorithm may be unjustifiable.
Even if a brute-force algorithm is too inefficient in
general, it may still be useful for solving small-size
instances of a problem. For theoretical / educational purpose. A brute-
force algorithm can be used as a benchmark for
comparing other efficient alternatives.
-
8/4/2019 03 Brute Force
4/33
3-4Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Brute-Force Sorting AlgorithmBrute-Force Sorting Algorithm
Selection SortSelection Sort Scan the array to find its smallest element andScan the array to find its smallest element andswap it with the first element. Then, starting with the secondswap it with the first element. Then, starting with the second
element, scan the elements to the right of it to find the smallestelement, scan the elements to the right of it to find the smallest
among them and swap it with the second elements. Generally,among them and swap it with the second elements. Generally,
on passon pass ii(0(0 ii n-n-2), find the smallest element in2), find the smallest element inAA[[i..n-i..n-1]1]and swap it withand swap it withAA[[ii]:]:
AA[0][0] . . .. . . AA[[ii-1] |-1] | AA[[ii], . . . ,], . . . ,AA[[minmin], . . .,], . . .,AA[[nn-1]-1]
in their final positionsin their final positions
Example: 7 3 2 5Example: 7 3 2 5
-
8/4/2019 03 Brute Force
5/33
3-5Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Analysis of Selection SortAnalysis of Selection Sort
Time efficiency:Time efficiency:
Space efficiency:Space efficiency:
Stability:Stability:
SIMULATION
-
8/4/2019 03 Brute Force
6/33
3-6Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
)()(
2)1()1(
2)2()1(2
2
)1)(2()1)(1(
2
)1)(2(1)1(
)1()1(
]1)1()1[(1)(
2
2
0
2
0
2
0
2
0
2
0
2
0
1
1
nnC
nnnnn
nnnn
nnn
inin
innC
n
i
n
i
n
i
n
i
n
i
n
i
n
ij
==
=
=
==
++==
=
=
=
=
=
=
+=
-
8/4/2019 03 Brute Force
7/333-7Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Bubble SortBubble Sort
a
Bubble sort is another example of applying brute-force toBubble sort is another example of applying brute-force tothe sorting algorithm.the sorting algorithm.
a Read section on Bubble sort in the textbook (page 100).Read section on Bubble sort in the textbook (page 100).
-
8/4/2019 03 Brute Force
8/33
3-8Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Applying Brute-Force to the SearchingApplying Brute-Force to the Searching
ProblemProblem
a
Read section on Sequential Search in your textbook (pageRead section on Sequential Search in your textbook (page103).103).
-
8/4/2019 03 Brute Force
9/33
3-9Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Brute-Force String MatchingBrute-Force String Matching
a patternpattern: a string of: a string ofmm characters to search forcharacters to search for
a texttext: a (longer) string of: a (longer) string ofnn characters to search incharacters to search in
a problem: find a substring in the text that matches the patternproblem: find a substring in the text that matches the pattern
Brute-force algorithmBrute-force algorithmStep 1 Align pattern at beginning of textStep 1 Align pattern at beginning of text
Step 2 Moving from left to right, compare each character ofStep 2 Moving from left to right, compare each character of
pattern to the corresponding character in text untilpattern to the corresponding character in text until
all characters are found to match (successful search); orall characters are found to match (successful search); or a mismatch is detecteda mismatch is detected
Step 3 While pattern is not found and the text is not yetStep 3 While pattern is not found and the text is not yet
exhausted, realign pattern one position to the right andexhausted, realign pattern one position to the right and
repeat Step 2repeat Step 2
-
8/4/2019 03 Brute Force
10/33
3-10Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Examples of Brute-Force String MatchingExamples of Brute-Force String Matching
1.1. Pattern:Pattern: 001011001011Text:Text: 1001010110100110010111101010010101101001100101111010
2.2. Pattern:Pattern: happyhappyText:Text: It is never too late to have a happyIt is never too late to have a happy
childhood.childhood.
SIMULATION
-
8/4/2019 03 Brute Force
11/33
3-11Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Pseudocode and EfficiencyPseudocode and Efficiency
Efficiency:Efficiency:
-
8/4/2019 03 Brute Force
12/33
3-12Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a
In the worst case,In the worst case,
a It has been shown that the average case efficiencyIt has been shown that the average case efficiency
is linear (Consider typical word search in ais linear (Consider typical word search in a
natural language text).natural language text).
)()(
)1()( 2
mnnC
mmmnmmnnC
worst
worst
+=+=
)()()( nmnnCaverage =+=
-
8/4/2019 03 Brute Force
13/33
3-13Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Brute-Force Polynomial EvaluationBrute-Force Polynomial Evaluation
Problem: Find the value of polynomialProblem: Find the value of polynomialpp((xx) =) = aannxx
nn++ aann-1-1xxnn-1-1 + ++ + aa11xx
11 ++ aa00
at a pointat a pointxx==xx00
Brute-force algorithmBrute-force algorithm
Efficiency:Efficiency:
pp0.00.0
forforiinndowntodownto 00 dodo
powerpower11
forforjj11 totoiidodo //compute//computexxii
powerpowerpowerpowerxx
pppp ++ aa[[ii]] powerpowerreturnreturnpp
-
8/4/2019 03 Brute Force
14/33
3-14Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
)(2
)1(
)11(1)(
2
0
00 1
nnni
inC
n
i
n
i
n
i
i
j
+==
+==
=
== =
-
8/4/2019 03 Brute Force
15/33
3-15Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Polynomial Evaluation: ImprovementPolynomial Evaluation: Improvement
We can do better by evaluatingWe can do better by evaluating
from right to left:from right to left:
Better brute-force algorithmBetter brute-force algorithm
Efficiency:Efficiency:
ppaa[0][0]
powerpower11forforii11 totonndodo
powerpowerpowerpower xx
pppp ++ aa[[ii]] powerpower
returnreturnpp
)()( nnC
-
8/4/2019 03 Brute Force
16/33
3-16Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Closest-Pair ProblemClosest-Pair Problem
Find the two closest points in a set ofFind the two closest points in a set ofnn points (in the two-points (in the two-dimensional Cartesian plane).dimensional Cartesian plane).
Brute-force algorithmBrute-force algorithm
Compute the distance between every pair of distinct pointsCompute the distance between every pair of distinct points
and return the indexes of the points for which the distanceand return the indexes of the points for which the distance
is the smallest.is the smallest.
-
8/4/2019 03 Brute Force
17/33
3-17Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Closest-Pair Brute-Force Algorithm (cont.)Closest-Pair Brute-Force Algorithm (cont.)
Efficiency:Efficiency:
How to make it faster?How to make it faster?
-
8/4/2019 03 Brute Force
18/33
3-18Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a The algorithm can be made faster by avoiding computing squareThe algorithm can be made faster by avoiding computing square
roots. Is it possible for this problem?roots. Is it possible for this problem?
a ReplaceReplace
withwith
))()(( 22 jiji yyxxsqrtd +
22 )()( jiji yyxxd +
-
8/4/2019 03 Brute Force
19/33
3-19Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
)()1(
)2
)1()1((2
)2
)1)1)((1()11)1(((2
)(2)(2
)1)1((2
122)(
2
1
1
1
1
1
1
1
1
1
1 1
1
1 1
nnn
nnnn
nnnn
inin
in
nC
n
i
n
i
n
i
n
i
n
i
n
ij
n
i
n
ij
=
=
++=
==
++=
==
=
=
=
=
= +=
= +=
-
8/4/2019 03 Brute Force
20/33
3-20Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Brute-Force Strengths and WeaknessesBrute-Force Strengths and Weaknesses
a StrengthsStrengths
wide applicabilitywide applicability
simplicitysimplicity
yields reasonable algorithms for some important problemsyields reasonable algorithms for some important problems(e.g., matrix multiplication, sorting, searching, string(e.g., matrix multiplication, sorting, searching, string
matching)matching)
a WeaknessesWeaknesses
rarely yields efficient algorithmsrarely yields efficient algorithms
some brute-force algorithms are unacceptably slowsome brute-force algorithms are unacceptably slow not as constructive as some other design techniquesnot as constructive as some other design techniques
-
8/4/2019 03 Brute Force
21/33
3-21Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Exhaustive SearchExhaustive Search
A brute force solution to a problem involving search for anA brute force solution to a problem involving search for an
element with a special property, usually among combinatorialelement with a special property, usually among combinatorialobjects such as permutations, combinations, or subsets of aobjects such as permutations, combinations, or subsets of a
set.set.
Method:Method: generate a list of all potential solutions to the problem in agenerate a list of all potential solutions to the problem in a
systematic manner (see algorithms in Sec. 5.4)systematic manner (see algorithms in Sec. 5.4)
evaluate potential solutions one by one, disqualifyingevaluate potential solutions one by one, disqualifyinginfeasible ones and, for an optimization problem, keepinginfeasible ones and, for an optimization problem, keepingtrack of the best one found so fartrack of the best one found so far
when search ends, announce the solution(s) foundwhen search ends, announce the solution(s) found
-
8/4/2019 03 Brute Force
22/33
3-22Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Example 1: Traveling Salesman ProblemExample 1: Traveling Salesman Problem
a
GivenGiven nn cities with known distances between each pair, findcities with known distances between each pair, findthe shortest tour that passes through all the cities exactlythe shortest tour that passes through all the cities exactly
once before returning to the starting cityonce before returning to the starting city
a Alternatively: Find shortestAlternatively: Find shortestHamiltonian circuitHamiltonian circuit in ain a
weighted connected graphweighted connected graph
a Example:Example:
a b
c d
8
2
7
5 3
4
-
8/4/2019 03 Brute Force
23/33
3-23Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
TSP by Exhaustive SearchTSP by Exhaustive Search
Tour CostTour Cost
aabbccdda 2+3+7+5 = 17a 2+3+7+5 = 17
aabbddcca 2+4+7+8 = 21a 2+4+7+8 = 21
aaccbbdda 8+3+4+5 = 20a 8+3+4+5 = 20
aaccddbba 8+7+4+2 = 21a 8+7+4+2 = 21aaddbbcca 5+4+3+8 = 20a 5+4+3+8 = 20
aaddccbba 5+7+3+2 = 17a 5+7+3+2 = 17
More tours?More tours?
Less tours?Less tours?
Only need to consider circuits starting from onevertex.
Only need to consider one tour direction.
E.g. No need to consideradcbaadcba
-
8/4/2019 03 Brute Force
24/33
3-24Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a Efficiency:Efficiency:
Consider starting from the same vertex and only 4 verticesConsider starting from the same vertex and only 4 vertices
in graphin graph
vv00vv11vv22vv33vv00
3 X 2 X 1 = 6 tours3 X 2 X 1 = 6 tours
If we consider only one direction thenIf we consider only one direction then
(3 X 2 X 1) / 2 = 3 tours(3 X 2 X 1) / 2 = 3 tours
-
8/4/2019 03 Brute Force
25/33
3-25Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a Efficiency:Efficiency:
So, for n vertices?So, for n vertices?
If we do not limit to starting from one vertex?If we do not limit to starting from one vertex?
and consider both directions?and consider both directions?
2
)!1(
2
1*2*...*)2(*)1( =
nnn
2
!
2
1*2*...*)2(*)1(* nnnn=
!2*2
!n
n=
-
8/4/2019 03 Brute Force
26/33
3-26Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Example 2: Knapsack ProblemExample 2: Knapsack Problem
GivenGiven nn items:items:
weights:weights: ww11 ww22 w wnn
values:values: vv11 vv22 v vnn
a knapsack of capacitya knapsack of capacity WW
Find most valuable subset of the items that fit into the knapsackFind most valuable subset of the items that fit into the knapsack
Example: Knapsack capacity W=16Example: Knapsack capacity W=16
item weight valueitem weight value
1 2 $201 2 $20
2 5 $302 5 $30
3 10 $503 10 $50
4 5 $104 5 $10
-
8/4/2019 03 Brute Force
27/33
3-27Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Knapsack Problem by Exhaustive SearchKnapsack Problem by Exhaustive Search
SubsetSubset Total weightTotal weight Total valueTotal value{1} 2 $20{1} 2 $20
{2} 5 $30{2} 5 $30
{3} 10 $50{3} 10 $50
{4} 5 $10{4} 5 $10
{1,2} 7 $50{1,2} 7 $50
{1,3} 12 $70{1,3} 12 $70{1,4} 7 $30{1,4} 7 $30
{2,3} 15 $80{2,3} 15 $80
{2,4} 10 $40{2,4} 10 $40
{3,4} 15 $60{3,4} 15 $60
{1,2,3} 17 not feasible{1,2,3} 17 not feasible{1,2,4} 12 $60{1,2,4} 12 $60
{1,3,4} 17 not feasible{1,3,4} 17 not feasible
{2,3,4} 20 not feasible{2,3,4} 20 not feasible
{1,2,3,4} 22 not feasible{1,2,3,4} 22 not feasible
-
8/4/2019 03 Brute Force
28/33
3-28Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Each subset of a set ofn elements {e0, e1, , en-1} can be
represented as a binary number ofn-1 bits.
For example, consider the set { 3, 5, 2, 8, 9 }. The binary
number1 0 0 1 0
represents the subset {3, 8 }.
The number of subsets for a set ofn elements is 2n.
Hence,)2()( nnC =
a Efficiency:Efficiency:
-
8/4/2019 03 Brute Force
29/33
3-29Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a
The Travelling Salesman and Knapsack problems areThe Travelling Salesman and Knapsack problems areexamples ofexamples ofNP-hard problemsNP-hard problems..
a Computer scientists believe that polynomial-time algorithmsComputer scientists believe that polynomial-time algorithms
do not exists for such problemsdo not exists for such problems
There are more sophisticated approaches which solveThere are more sophisticated approaches which solvesome but not all instances of those problems in less thansome but not all instances of those problems in less than
exponential time.exponential time.
Another approach is to use approximation algorithms.Another approach is to use approximation algorithms.
-
8/4/2019 03 Brute Force
30/33
3-30Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
Example 3: The Assignment ProblemExample 3: The Assignment Problem
There areThere are nn people who need to be assigned topeople who need to be assigned to nn jobs, one personjobs, one person
per job. The cost of assigning personper job. The cost of assigning person iito jobto jobjjis C[is C[ii,,jj]. Find an]. Find anassignment that minimizes the total cost.assignment that minimizes the total cost.
Job 0 Job 1 Job 2 Job 3Job 0 Job 1 Job 2 Job 3
Person 0 9Person 0 9 2 7 82 7 8
Person 1 6 4 3 7Person 1 6 4 3 7
Person 2 5 8 1 8Person 2 5 8 1 8
Person 3 7 6 9 4Person 3 7 6 9 4
Algorithmic Plan:Algorithmic Plan:Generate all legitimate assignments, computeGenerate all legitimate assignments, computetheir costs, and select the cheapest one.their costs, and select the cheapest one.
How many assignments are there?How many assignments are there?
Pose the problem as the one about a cost matrix:Pose the problem as the one about a cost matrix:
-
8/4/2019 03 Brute Force
31/33
3-31Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
9 2 7 89 2 7 8
6 4 3 76 4 3 7 5 8 1 85 8 1 8
7 6 9 47 6 9 4
AssignmentAssignment (col.#s)(col.#s) Total CostTotal Cost
1, 2, 3, 41, 2, 3, 4 9+4+1+4=189+4+1+4=18
1, 2, 4, 31, 2, 4, 3 9+4+8+9=309+4+8+9=30
1, 3, 2, 41, 3, 2, 4 9+3+8+4=249+3+8+4=24
1, 3, 4, 21, 3, 4, 2 9+3+8+6=269+3+8+6=26
1, 4, 2, 31, 4, 2, 3 9+7+8+9=339+7+8+9=331, 4, 3, 21, 4, 3, 2 9+7+1+6=239+7+1+6=23
etc.etc.
Assignment Problem by Exhaustive SearchAssignment Problem by Exhaustive Search
C =C =
Generate all
permutations
-
8/4/2019 03 Brute Force
32/33
3-32Copyright 2007 Pearson Addison-Wesley. All rights reserved.A. Levitin Introduction to the Design & Analysis of Algorithms, 2nd ed., Ch. 3
a
Number of permutations to be considered isNumber of permutations to be considered is n!.n!. This problems domain grows exponentially; exhaustiveThis problems domain grows exponentially; exhaustive
search is impratical (except for very small instances of thesearch is impratical (except for very small instances of the
problem)problem)
a
Actually, there is a much more efficient algorithm for thisActually, there is a much more efficient algorithm for thisproblem called theproblem called theHungarian methodHungarian method..
a It is important to remember that, more often than not, thereIt is important to remember that, more often than not, there
are no known polynomial-time algorithms for problemsare no known polynomial-time algorithms for problems
whose domain grows exponentially with instance size.whose domain grows exponentially with instance size.
-
8/4/2019 03 Brute Force
33/33
3 33A L iti I t d ti t th D i & A l i f Al ith 2 d d Ch 3
Final Comments on Exhaustive SearchFinal Comments on Exhaustive Search
a
Exhaustive-search algorithms run in a realistic amount ofExhaustive-search algorithms run in a realistic amount oftimetime only on very small instancesonly on very small instances
a In some cases, there are much better alternatives!In some cases, there are much better alternatives!
Euler circuitsEuler circuits shortest pathsshortest paths
minimum spanning treeminimum spanning tree
assignment problemassignment problem
a In many cases, exhaustive search or its variation is the onlyIn many cases, exhaustive search or its variation is the only
known way to get exact solutionknown way to get exact solution