today: optimal binary search
TRANSCRIPT
Today: β Optimal Binary Search
COSC 581, Algorithms January 30, 2014
Many of these slides are adapted from several online sources
Reading Assignments
β’ Todayβs class: β Chapter 15.5
β’ Reading assignment for next class:
β Chapter 22, 24.0, 24.2, 24.3
β’ Announcement: Exam 1 is on Tues, Feb. 18 β Will cover everything up through dynamic
programming
Optimal Binary Search Trees β’ Problem Statement:
β Given sequence πΎ = π1 < π2 < β― < ππ of n sorted keys, with a search probability ππ for each key ππ.
β Also given π + 1 βdummy keysβ π0,π1, β¦ , ππrepresenting searches not in ππ β’ In particular, d0 represents all values less than π1, ππ represents all values
greater than ππ, and for π = 1, 2, β¦ ,π β 1, the dummy key ππ represents all values between ππ and ππ+1.
β’ The dummy keys are leaves (external nodes), and the data keys are internal nodes.
β’ For each dummy key ππ, we have search probability ππ
β We want to build a binary search tree (BST) with minimum expected search cost.
β Actual cost = # of items examined. β For key ππ, cost = depthT(ππ)+1, where depthT(ππ) = depth of ππ in BST T .
We add 1 because root is at depth 0
Expected Search Cost
β β= =
=+n
i
n
iii qp
1 01
β β
β β
= =
= =
β +β +=
β ++β +=
n
ii
n
iiTiiT
i
n
i
n
iiTiiT
qdpk
qdpk
1 0
1 0
)(depth)(depth1
)1)(depth()1)(depth(T]in cost E[search
Since every search is either successful or not, the probabilities sum to 1:
Example β Expected Search Cost
cost: 2.80
Example β Expected Search Cost
cost: 2.80 cost: 2.75 optimal!!
But thereβs a better solution:
Observations
β’ Observations: β Optimal BST may not have smallest height. β Optimal BST may not have highest-probability key
at root.
Brute Force?
β’ Build by exhaustive checking? β Construct each n-node BST. β For each,
assign keys and compute expected search cost. β Pick best
β’ But there are Ξ©(4n/n3/2) different BSTs with n nodes!
Step 1: Optimal Substructure β’ Any subtree of a BST contains keys in a contiguous range ππ, ..., ππ for some 1 β€ π β€ π β€ π.
β’ If T is an optimal BST and T contains subtree Tβ² with keys ππ, ..., ππ, then Tβ² must be an optimal BST for keys ππ, ..., ππ.
β’ Proof: Cut and paste.
T
Tβ²
Optimal Substructure
β’ One of the keys in ππ, ..., ππ, say ππ, where π β€ r β€ π, must be the root of an optimal subtree for these keys.
β’ Left subtree of ππ contains ππ,...,krβ1. β’ Right subtree of ππ contains ππ+1 , ..., ππ.
β’ To find an optimal BST: β Examine all candidate roots ππ , for π β€ r β€ π β Determine all optimal BSTs containing ππ,...,krβ1 and
containing kr+1,..., ππ
kr
ki kr-1 kr+1 kj
Step 2: Recursive Solution
β’ Find optimal BST for ππ, ..., ππ, where π β₯ 1, π β€ π, π β₯ πβ1. When π = πβ1, the tree is empty.
β’ Define π[π, π ] = expected search cost of optimal BST for ππ, β¦ ,ππ.
β’ If π = π β 1, then π[π, π ] = ππβ1. β’ If π β₯ π,
β Select a root ππ for some π β€ π β€ π . β Recursively make optimal BSTs
β’ for ππ,...,krβ1 as the left subtree, and β’ for ππ+1 , ..., ππ as the right subtree.
Step 2: Recursive Solution β’ When the optimal subtree becomes a subtree of a node:
β Depth of every node in optimal subtree goes up by 1. β Expected search cost increases by sum of probabilities of
subtree:
β’ If ππ is the root of an optimal BST for ππ, . . , ππ : π[π, π ] = ππ + (π[π, πβ1] + π€(π, πβ1)) + (π[π + 1, π] + π€(π + 1, π)) = π[π, πβ1] + π[π + 1, π] + π€(π, π).
β’ But, we donβt know kr. Hence,
βββ==
+=j
ill
j
ill qpjiw
1),(
(because π€(π, π) = π€(π, πβ1) + ππ + π€(π + 1, π))
π π, π = ππβ1 if π = π β 1
minπβ€πβ€π
π π, π β 1 + π π + 1, π + π€(π, π) if π β€ π
Step 3: Computing an Optimal Solution
For each subproblem (π, π), store: β’ Expected search cost in a table π[1 . .π + 1 , 0 . .π]
β Will use only entries π[π, π ], where π β₯ πβ1.
β’ root[π, π] = root of subtree with keys ππ, . . , ππ, for 1 β€ π β€ π β€ π.
β’ π€[1. .π + 1, 0. .π] = sum of probabilities: β π€[π, πβ1] = ππβ1 for 1 β€ π β€ π + 1 (base case) β π€[π, π ] = π€[π, π β 1] + ππ + ππ for 1 β€ π β€ π β€ π
Note: w is stored for purposes of efficiency. Rather than compute π€(π, π) from scratch every time we compute π π, π , we store these values in a table instead.
Step 3: Computing the expected search cost of an optimal binary search tree
Ch. 15 Dynamic Programming
Consider all trees with π keys. Fix the first key. Fix the last key
Determine the root of the optimal (sub)tree
Time?
Step 3: Computing the expected search cost of an optimal binary search tree
Ch. 15 Dynamic Programming
Consider all trees with π keys. Fix the first key. Fix the last key
Determine the root of the optimal (sub)tree
Time = π(π3)
Table π[π, π], π€[π, π], and ππππ[π, π] computed by OPTIMAL-BST on an example key distribution:
Dynamic Programming Exercise β’ Consider an exam with π questions. For each π = 1, β¦ ,π, question π has
integral point value π£π > 0 and requires ππ > 0 minutes to solve. Suppose further that no partial credit is awarded.
β’ The ultimate goal would be to come up with an algorithm which, given π£1, π£2, β¦ , π£π,π1,π2, β¦ ,ππ, and V, computes the minimum number of minutes required to earn at least V points on the exam. (For example, you might use this algorithm to determine how quickly you can get an A on the exam.)
β’ Let π(π, π£) denote the minimum number of minutes needed to earn π£ points when you are restricted to selecting from questions 1 through π. Complete the following recurrence expression for π(π, π£) (i.e., fill in the blank). The base cases are supplied for you. 0 for all π, if π£ β€ 0 π(π, π£) = β if π = 0 and π£ > 0 otherwise
Another DP Exercise [In this problem, we define meanings for upper-case Sk and W, as well as for lower-case sk and w. Keep in mind that the meanings of these variables are case-sensitive β i.e., sk is not the same thing as Sk, and W does not have the same meaning as w. These are defined clearly below.] The 0-1 Knapsack problem is as follows. You are given a knapsack with maximum capacity W and a set S consisting of π items, π 1, π 2, β¦ , π π. Each item π π has some weight π€π and benefit value ππ. Here, all π€π, ππ, and W are integer values. The problem is to pack the knapsack so as to achieve the maximum total value of the packed items. Each item has to be either entirely accepted, or entirely rejected; no partial items are allowed. 1. In a brute force approach, we would search all possible combinations of items and find the best one. How many possible combinations would we have to search using this brute force approach? (State your answer in big-O notation.)
Another DP Exercise Let us now consider a dynamic programming approach to this problem. We will define subproblems as follows. Let ππ be the subset of items π 1, π 2, β¦ , π π. We want to find the optimal solution for ππ, which would be the subset of items in ππ that achieves the maximum total value. To make this work properly, we need to define another parameter w, which is the exact weight for each subset of items. Thus, we now define a subproblem to be to compute B[k, w], which represents the value of the best subset of ππ that has total weight of exactly w. We can now define a recursive solution for B[k, w] that considers 2 choices β either item sk is part of the solution or it isn't. (We also must include a base case.) What is this recursive solution ?
β’ π΅ π,π€ =
if π€π > π€
otherwise
Another DP Exercise Let us now consider a dynamic programming approach to this problem. We will define subproblems as follows. Let ππ be the subset of items π 1, π 2, β¦ , π π. We want to find the optimal solution for ππ, which would be the subset of items in ππ that achieves the maximum total value. To make this work properly, we need to define another parameter w, which is the exact weight for each subset of items. Thus, we now define a subproblem to be to compute B[k, w], which represents the value of the best subset of ππ that has total weight of exactly w. We can now define a recursive solution for B[k, w] that considers 2 choices β either item sk is part of the solution or it isn't. (We also must include a base case.) What is this recursive solution ?
In filling in this B[k, w] table, what are the indices over which k runs? In filling in this B[k, w] table, what are the indices over which w runs? How much work has to be done for each subproblem (stated in Ξ notation)? What is the runtime of the DP alg. that implements this recursive solution?
Reading Assignments
β’ Reading assignment for next class: β Chapter 22, 24.0, 24.2, 24.3
β’ Announcement: Exam 1 is on Tues, Feb. 18 β Will cover everything up through dynamic
programming