Chapter 5.Chapter 5.

Probabilistic Analysis Probabilistic Analysis and Randomized and Randomized

AlgorithmsAlgorithms

OutlineOutline

Explain the Explain the differencesdifferences between between probabilisticprobabilistic analysisanalysis and and randomized algorithmsrandomized algorithms

Present the technique of Present the technique of indicator random indicator random variablesvariables

Give another example of the analysis of a Give another example of the analysis of a randomized algorithm (permuting an array in randomized algorithm (permuting an array in place)place)

The hiring problemThe hiring problem

The hiring problem The hiring problem O(ncO(ncii+mc+mchh))- - Interview & hireInterview & hire

- Point: after interviewing candidate i, determine if - Point: after interviewing candidate i, determine if candidate i is the best candidate you have seen so farcandidate i is the best candidate you have seen so far

Worst-case analysisWorst-case analysis - - We hire every candidate that we interviewWe hire every candidate that we interview

- Hiring cost: - Hiring cost: O(ncO(nchh))

- - We neither have any idea nor have any control over the We neither have any idea nor have any control over the order of candidatesorder of candidates

What we expect to happen in a What we expect to happen in a typical typical or averageor average case? case?

Probabilistic Analysis(1)Probabilistic Analysis(1)

The use of probability in the analysis of The use of probability in the analysis of problemsproblems

Use knowledge (Use knowledge (make assumptionmake assumption) about ) about the distribution of the inputsthe distribution of the inputs

ExpectedExpected cost/running time cost/running time

Not applicable to all problemsNot applicable to all problems

Random orderRandom order (imply a total order)-> (imply a total order)-> uniform random permutation: n!uniform random permutation: n!

Probabilistic Analysis(2)Probabilistic Analysis(2)

Randomized Randomized Algorithms(1)Algorithms(1)

Algorithms that make random decisionsAlgorithms that make random decisionsThat isThat is– Can generate a random number x from some range Can generate a random number x from some range

{1, 2…R}.{1, 2…R}.– Make decisions based on the value of xMake decisions based on the value of x– Instead of Instead of guessingguessing whether the order is in a whether the order is in a

random order, we random order, we impose impose a random ordera random order– Its behavior is determined not only by its input but Its behavior is determined not only by its input but

also by values produced by a random-number also by values produced by a random-number generatorgenerator

Randomized Randomized Algorithms(2)Algorithms(2)

Indicator Random Indicator Random Variables(1)Variables(1)

Indicator Random Indicator Random Variables(2)Variables(2)

Provide a convenient method for converting Provide a convenient method for converting between probabilities and between probabilities and expectationsexpectations

Sample space and event A, associated with a Sample space and event A, associated with a indicator random variableindicator random variable I{A}I{A}

LemmaLemma::

- - The The expected valueexpected value of an indicator random variable of an indicator random variable associated with an event Aassociated with an event A is equal to the is equal to the probabilityprobability that A occursthat A occurs

Is very useful and convenient for analyzing Is very useful and convenient for analyzing situations in which we perform situations in which we perform repeated trialsrepeated trials

Analysis of the hiring Analysis of the hiring problem using indicator problem using indicator

random variablerandom variableAssume the candidates arrive in a random Assume the candidates arrive in a random orderorder

X: X: number of times we hire a new office assistantnumber of times we hire a new office assistant

X=XX=X11 +X +X22+…+X+…+Xnn--XXii={={candidate i is hiredcandidate i is hired}}

E[XE[Xii]=Pr {candidate i is hired}=]=Pr {candidate i is hired}=1/i1/i

E[X]=lnE[X]=lnnn++O(1)O(1)

Conclusion: Even though we interview n candidates, Conclusion: Even though we interview n candidates, we only actually hire lnwe only actually hire lnnn of them of them

Hiring ProblemHiring Problem

Lemma 5.2Lemma 5.2

Assuming that the candidates are presented in a Assuming that the candidates are presented in a random order, algorithm HIRE has a total hiring random order, algorithm HIRE has a total hiring cost of O(cost of O(CChhlnlnnn))

Worst-case Running timeWorst-case Running time

The worst-case hiring cost is O(The worst-case hiring cost is O(nCnChh). The ). The

expected interview cost is a significant expected interview cost is a significant improvement over the worst-case cost improvement over the worst-case cost

Random Algorithms Random Algorithms Explore the distinction between probabilistic analysis and Explore the distinction between probabilistic analysis and randomized algorithmsrandomized algorithmsInstead of assuming a distribution, we Instead of assuming a distribution, we imposeimpose a a distributiondistributionFor probabilistic analysis, the algorithm is For probabilistic analysis, the algorithm is deterministicdeterministic..The number of times we hire differs for different inputs The number of times we hire differs for different inputs (expensive, inexpensive and moderately expensive inputs)(expensive, inexpensive and moderately expensive inputs)For randomized algorithm, the For randomized algorithm, the randomizationrandomization is in the is in the algorithm not in the algorithm not in the input distributioninput distributionFor a given input, each time we run the randomized For a given input, each time we run the randomized algorithm, we get algorithm, we get different updatesdifferent updates so as to get different so as to get different costcostNo particular input elicits its No particular input elicits its worst-case behaviorworst-case behavior

Randomly Permuting arraysRandomly Permuting arrays

Randomize the input by Randomize the input by permutingpermuting the the given input arraygiven input array

Assign each element of the array a Assign each element of the array a random priority random priority p[i]p[i] and then sort the array and then sort the array according to these prioritiesaccording to these priorities

Randomly permuting produces a uniform Randomly permuting produces a uniform random permutationrandom permutation

Permute the given array in placePermute the given array in place

Permute-By-SortingPermute-By-Sorting

Permute-By-Sorting (A)1 n length [A]2 for i 1 to n3. do P [i] =RANDOM (1, n3)4. sort A, using P as sort keys5. return A

Procedure Permute-By-Sorting produces a uniform random permutation of the input in

(nlgn)(nlgn), assume all priorities are unique

1 2 3 1

1 2 1 3 1 2

1 2 3 1

1 2 3 1

Pr( ... )

Pr( ) Pr( | ) Pr( | )

Pr( | ... )

Pr( | ... )

1 1 1 11

1 2 !

n n

i i

n n

X X X X X

X X X X X X

X X X X X

X X X X X

n n n n

There are total n! permutations and each permutation appears with 1/n! probability.

Randomize in PlaceRandomize in Place1. n length [A]2. for i 1 to n3. do swap A [i] A[RANDOM (i,n)]

! !! !

!( )! ( )!

( 1)...( 1)

kn

n nC k k

k n k n k

n n n k

Proof of Lemma 5.5Proof of Lemma 5.5

HomeworkHomework

5.1-2,5.2-45.1-2,5.2-4

5.3-3,5.3-45.3-3,5.3-4