greedy algorithms 15-211 fundamental data structures and algorithms peter lee march 19, 2004
TRANSCRIPT
![Page 1: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/1.jpg)
Greedy Algorithms
15-211 Fundamental Data Structures and Algorithms
Peter Lee
March 19, 2004
![Page 2: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/2.jpg)
Announcements
• HW6 is due on April 5!
• Quiz #2 postponed until March 31 an online quiz requires up to one hour of uninterrupted
time with a web browser• actually, only a 15-minute quiz
must be completed by April 1, 11:59pm
![Page 3: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/3.jpg)
![Page 4: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/4.jpg)
Objects in calendar are closer than they appear
![Page 5: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/5.jpg)
Greed is Good
![Page 6: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/6.jpg)
Example: Counting change
• Suppose we want to give out change, using the minimal number of bills and coins.
![Page 7: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/7.jpg)
A change-counting algorithm
• An easy algorithm for giving out N cents in change:
Choose the largest bill or coin that is N. Subtract the value of the chosen bill/coin from N,
to get a new value of N. Repeat until a total of N cents has been counted.
• Does this work? I.e., does this really give out the minimal number of coins and bills?
![Page 8: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/8.jpg)
Our simple algorithm
• For US currency, this simple algorithm actually works.
• Why do we call this a greedy algorithm?
![Page 9: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/9.jpg)
Greedy algorithms
• At every step, a greedy algorithm
makes a locally optimal decision,
with the idea that in the end it all adds up to a globally optimal solution.
• Being optimistic like this usually leads to very simple algorithms.
![Page 10: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/10.jpg)
Lu Lu’s Pan Fried Noodle Shop
Think Globally
Act Locally
Eat Noodles
Over on Craig Street…
How Californian...
![Page 11: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/11.jpg)
But…
• What happens if we have a 12-cent coin?
![Page 12: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/12.jpg)
Hill-climbing
• Greedy algorithms are often visualized as “hill-climbing”. Suppose you want to reach the summit,
but can only see 10 yards ahead and behind (due to thick fog).
Which way?
![Page 13: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/13.jpg)
Hill-climbing
• Greedy algorithms are often visualized as “hill-climbing”. Suppose you want to reach the summit,
but can only see 10 yards ahead and behind (due to thick fog).
Which way?
![Page 14: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/14.jpg)
Hill-climbing, cont’d
• Making the locally-best guess is efficient and easy, but doesn’t always work.
![Page 15: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/15.jpg)
Where have we seen this before?
• Greedy algorithms are common in computer science
• In fact, from last week…
![Page 16: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/16.jpg)
Finding shortest airline routes
PVD
BOS
JFK
ORD
LAX
SFO
DFWBWI
MIA
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 17: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/17.jpg)
Three 2-hop BWI->DFW routes
PVD
BOS
JFK
ORD
LAX
SFO
DFWBWI
MIA
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 18: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/18.jpg)
A greedy algorithm
• Assume that every city is infinitely far away. I.e., every city is miles away from BWI
(except BWI, which is 0 miles away). Now perform something similar to
breadth-first search, and optimistically guess that we have found the best path to each city as we encounter it.
If we later discover we are wrong and find a better path to a particular city, then update the distance to that city.
![Page 19: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/19.jpg)
Intuition behind Dijkstra’s alg.
• For our airline-mileage problem, we can start by guessing that every city is miles away. Mark each city with this guess.
• Find all cities one hop away from BWI, and check whether the mileage is less than what is currently marked for that city. If so, then revise the guess.
• Continue for 2 hops, 3 hops, etc.
![Page 20: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/20.jpg)
Shortest mileage from BWI
PVD
BOS
JFK
ORD
LAX
SFO
DFW
BWI0
MIA
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 21: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/21.jpg)
Shortest mileage from BWI
PVD
BOS
JFK184
ORD621
LAX
SFO
DFW
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 22: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/22.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX
SFO
DFW1575
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 23: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/23.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX
SFO
DFW1575
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 24: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/24.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX
SFO3075
DFW1575
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 25: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/25.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 26: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/26.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX3288
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 27: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/27.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX2658
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 28: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/28.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX2658
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 29: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/29.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX2658
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 30: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/30.jpg)
Shortest mileage from BWI
PVD328
BOS371
JFK184
ORD621
LAX2658
SFO2467
DFW1423
BWI0
MIA946
337
2704
1846
1464
1235
2342
802
867
849
740
187
144
1391
184
1121946
1090
1258621
![Page 31: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/31.jpg)
Dijkstra’s algorithm
• Algorithm initialization:
Label each node with the distance , except start node, which is labeled with distance 0.• D[v] is the distance label for v.
Put all nodes into a priority queue Q, using the distances as labels.
![Page 32: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/32.jpg)
Dijkstra’s algorithm, cont’d
• While Q is not empty do: u = Q.removeMin for each node z one hop away from u do:
• if D[u] + miles(u,z) < D[z] then• D[z] = D[u] + miles(u,z)• change key of z in Q to D[z]
• Note use of priority queue allows “finished” nodes to be found quickly (in O(log N) time).
![Page 33: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/33.jpg)
Another Greedy Algorithm
![Page 34: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/34.jpg)
The Fractional Knapsack Problem (FKP)
• You rob a store: find n kinds of items Gold dust. Wheat. Beer.
![Page 35: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/35.jpg)
![Page 36: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/36.jpg)
Example 2: Fractional knapsack problem (FKP)
• You rob a store: find n kinds of items Gold dust. Wheat. Beer.
• The total inventory for the i th kind of item:
Weight: wi pounds Value: vi dollars
• Knapsack can hold a maximum of W pounds.
• Q: how much of each kind of item should you take?
(Can take fractional weight)
![Page 37: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/37.jpg)
FKP: solution
• Greedy solution: Fill knapsack with “most valuable” item
until all is taken. •Most valuable = vi /wi (dollars per pound)
Then next “most valuable” item, etc. Until knapsack is full.
![Page 38: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/38.jpg)
Ingredients of a greedy alg.
• An optimization problem.
• Is iterative / Proceeds in stages.
• Has the greedy-choice property:A greedy choice will lead to a globally optimal solution.
![Page 39: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/39.jpg)
FKP is greedy
• An optimization problem: Maximize value of loot, subject to
maximum weight W. (constrained optimization)
• Proceeds in stages: Knapsack is filled with one item at a
time.
![Page 40: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/40.jpg)
FKP is greedy
• Greedy-choice property: A locally greedy choice will lead to a globally optimal solution.
• In steps…:Step 1: Does the optimal solution contain the greedy choice?
Step 2: can the greedy choice always be made first?
![Page 41: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/41.jpg)
FKP: Greedy-choice: Step 1
• Consider total value, V, of knapsack.
• Knapsack must contain item h: Item h is the item with highest $/lb.
• Why? Because if h is not included, we can replace some other item in knapsack with an equivalent weight of h, and increase V.
• This can continue until knapsack is full, or all of h is taken.
• Therefore any optimal solution must include greedy-choice.
![Page 42: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/42.jpg)
More rigorously…
Let item h be the item with highest $/lb.Total inventory of h is wh pounds.Total value of h is vi dollars.
Let ki be weight of item i in knapsack. Then total value:
If kh<wh, and kj>0 for some jh, then replace j with an equal weight of h. Let new total value = V’.
Difference in total value:
since, by definition of h,
Therefore all of item h should be taken.
n
i i
ii w
vkV
1
0
j
jj
h
hj w
vk
w
vkVV
h
h
j
j
w
v
w
v
![Page 43: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/43.jpg)
FKP: Greedy-choice: Step 2
• Now we want to show that we can always make the greedy choice first.
• If item h is more than what knapsack can hold, then fill knapsack completely with h. No other item gives higher total value.
• Otherwise, knapsack contains h and some other item. We can always make h the first choice, without changing total value V.
• Therefore greedy-choice can always be made first.
![Page 44: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/44.jpg)
More rigorously…
• Case I: wh W Fill knapsack completely with h. No other item gives higher total value.
• Case II: wh < W Let 1st choice be item i, and kth choice be h,
then we can always swap our 1st and kth choices, and total value V remains unchanged.
• Therefore greedy-choice can always be made first.
![Page 45: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/45.jpg)
The Binary Knapsack Problem
• You win the Supermarket Shopping Spree contest. You are given a shopping cart with capacity
C. You are allowed to fill it with any items you
want from Giant Eagle. Giant Eagle has items 1, 2, … n, which have
values v1, v2, …, vn, and sizes s1, s2, …, sn. How do you (efficiently) maximize the value
of the items in your cart?
![Page 46: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/46.jpg)
![Page 47: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/47.jpg)
BKP is not greedy
• The obvious greedy strategy of taking the maximum value item that still fits in the cart does not work.
• Consider: Suppose item i has size si = C and value
vi. It can happen that there are items j and
k with combined size sj+sk C but vj+vk > vi.
![Page 48: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/48.jpg)
BKP: Greedy approach fails
item 1 item 2 item 3 knapsack
$60, 10 lbs
$100, 20 lbs
$120, 30 lbs
Maximum weight = 50 lbs
Dollars/pound
Item 1 $6
Item 2 $5
Item 3 $4
BKP has optimal substructure, but not greedy-choice property: optimal solution does not contain greedy choice.
$160 $180 $220
(optimal)
![Page 49: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/49.jpg)
A question for a future lecture…
• How can we (efficiently) solve the binary knapsack problem?
• One possible approach: Dynamic programming
![Page 50: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/50.jpg)
Machine Scheduling
![Page 51: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/51.jpg)
Optimal machine scheduling
• We are given n tasks and an infinite supply of machines to perform them each task ti = [si, fi] has start time si and
finish time fi
• An assignment of tasks to machines is feasible if no machine is assigned two overlapping tasks
• An assignment is optimal if it is feasible and uses the minimal number of machines
![Page 52: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/52.jpg)
Example
• Tasks: task: a b c d e f g start: 0 3 4 9 7 1 6 finish: 2 7 7 11 10 5 8
• Can you invent a greedy algorithm to find an optimal schedule for these tasks?
![Page 53: Greedy Algorithms 15-211 Fundamental Data Structures and Algorithms Peter Lee March 19, 2004](https://reader033.vdocuments.us/reader033/viewer/2022052510/56649e455503460f94b3a198/html5/thumbnails/53.jpg)
Succeeding with greed
3 ingredients needed:
• Optimization problem.
• Proceed in stages.
• Greedy-choice property:A greedy choice will lead to a globally optimal solution.